nixbot

1these 13 derivations will be built:2  /nix/store/82m58v5zssg437a0r0iz3cw33zf4nis5-python3.13-nixbot.drv3  /nix/store/7n7vrbdb75f3sfcylc46himhxj5jfh3c-unit-nixbot.service.drv4  /nix/store/rg9wjl11049132lpz4xw91vl8d5d3i9v-system-units.drv5  /nix/store/6qdj8rxk0ys9d4y5n6m5zmybp80f23g9-etc.drv6  /nix/store/ln8lcgjaq1w4cbhdpv1qxa1d4ikn05ms-activate.drv7  /nix/store/dwzwlfnj7q0d1xiwpbcssn8fy7l7s2fi-nixos-system-gitlab-test.drv8  /nix/store/5vavimybd82gfghgx6cvbbnkbqxdiwyn-closure-info.drv9  /nix/store/h73kxxssslbfjqzr9gcwkyvgy9k1pxkk-closure-info.drv10  /nix/store/8f7f5bb5njxxr8w4gny8zsna94697cgw-run-nixos-vm.drv11  /nix/store/jaw0j84cs3pbzyml6f99j07bxjh2r3lr-nixos-vm.drv12  /nix/store/6mxvfzdi3rjwfv0ia1i8xd4yh8s3g6ly-driverConfiguration.json.drv13  /nix/store/94a8j8fzj6kjxxbz7j6sf440rmgqv94h-nixos-test-driver-nixbot-gitlab.drv14  /nix/store/5693hy6nhi7cj4llpx19b2mgi54fysfz-vm-test-run-nixbot-gitlab.drv15building '/nix/store/82m58v5zssg437a0r0iz3cw33zf4nis5-python3.13-nixbot.drv' on 'ssh-ng://nix@jamie'16building '/nix/store/82m58v5zssg437a0r0iz3cw33zf4nis5-python3.13-nixbot.drv'17python3.13-nixbot> Sourcing python-remove-tests-dir-hook18python3.13-nixbot> Sourcing python-catch-conflicts-hook.sh19python3.13-nixbot> Sourcing python-remove-bin-bytecode-hook.sh20python3.13-nixbot> Sourcing pypa-build-hook21python3.13-nixbot> Using pypaBuildPhase22python3.13-nixbot> Sourcing python-runtime-deps-check-hook23python3.13-nixbot> Using pythonRuntimeDepsCheckHook24python3.13-nixbot> Sourcing pypa-install-hook25python3.13-nixbot> Using pypaInstallPhase26python3.13-nixbot> Sourcing python-imports-check-hook.sh27python3.13-nixbot> Using pythonImportsCheckPhase28python3.13-nixbot> Sourcing python-namespaces-hook29python3.13-nixbot> Sourcing python-catch-conflicts-hook.sh30python3.13-nixbot> Running phase: unpackPhase31python3.13-nixbot> unpacking source archive /nix/store/lnkaw1dn4l98zy1z3vjqnbrdvfba75hp-nixbot32python3.13-nixbot> source root is nixbot33python3.13-nixbot> setting SOURCE_DATE_EPOCH to timestamp 315619200 of file "nixbot/pyproject.toml"34python3.13-nixbot> Running phase: patchPhase35python3.13-nixbot> Running phase: updateAutotoolsGnuConfigScriptsPhase36python3.13-nixbot> Running phase: configurePhase37python3.13-nixbot> no configure script, doing nothing38python3.13-nixbot> Running phase: buildPhase39python3.13-nixbot> Executing pypaBuildPhase40python3.13-nixbot> Creating a wheel...41python3.13-nixbot> pypa build flags: --no-isolation --outdir dist/ --wheel42python3.13-nixbot> * Getting build dependencies for wheel...43python3.13-nixbot> * Building wheel...44python3.13-nixbot> Successfully built nixbot-0.0.1-py3-none-any.whl45python3.13-nixbot> Finished creating a wheel...46python3.13-nixbot> Finished executing pypaBuildPhase47python3.13-nixbot> Running phase: pythonRuntimeDepsCheckHook48python3.13-nixbot> Executing pythonRuntimeDepsCheck49python3.13-nixbot> Checking runtime dependencies for nixbot-0.0.1-py3-none-any.whl50python3.13-nixbot> Finished executing pythonRuntimeDepsCheck51python3.13-nixbot> Running phase: installPhase52python3.13-nixbot> Executing pypaInstallPhase53python3.13-nixbot> Successfully installed nixbot-0.0.1-py3-none-any.whl54python3.13-nixbot> Finished executing pypaInstallPhase55python3.13-nixbot> Running phase: pythonOutputDistPhase56python3.13-nixbot> Executing pythonOutputDistPhase57python3.13-nixbot> Finished executing pythonOutputDistPhase58python3.13-nixbot> Running phase: fixupPhase59python3.13-nixbot> shrinking RPATHs of ELF executables and libraries in /nix/store/b9495qi9g51hpmpvk2vbvyd4rxpapjkj-python3.13-nixbot60python3.13-nixbot> checking for references to /build/ in /nix/store/b9495qi9g51hpmpvk2vbvyd4rxpapjkj-python3.13-nixbot...61python3.13-nixbot> patching script interpreter paths in /nix/store/b9495qi9g51hpmpvk2vbvyd4rxpapjkj-python3.13-nixbot62python3.13-nixbot> stripping (with command strip and flags -S -p) in  /nix/store/b9495qi9g51hpmpvk2vbvyd4rxpapjkj-python3.13-nixbot/lib /nix/store/b9495qi9g51hpmpvk2vbvyd4rxpapjkj-python3.13-nixbot/bin63python3.13-nixbot> shrinking RPATHs of ELF executables and libraries in /nix/store/1lqcr680c69zw4v625dgag1x9dn4d0h2-python3.13-nixbot-dist64python3.13-nixbot> checking for references to /build/ in /nix/store/1lqcr680c69zw4v625dgag1x9dn4d0h2-python3.13-nixbot-dist...65python3.13-nixbot> patching script interpreter paths in /nix/store/1lqcr680c69zw4v625dgag1x9dn4d0h2-python3.13-nixbot-dist66python3.13-nixbot> Rewriting #!/nix/store/60m4rxhg2fldqaak400c0lry96ijrzqn-python3-3.13.13/bin/python3.13 to #!/nix/store/60m4rxhg2fldqaak400c0lry96ijrzqn-python3-3.13.1367python3.13-nixbot> wrapping `/nix/store/b9495qi9g51hpmpvk2vbvyd4rxpapjkj-python3.13-nixbot/bin/nixbot'...68python3.13-nixbot> Executing pythonRemoveTestsDir69python3.13-nixbot> Finished executing pythonRemoveTestsDir70python3.13-nixbot> Running phase: pythonCatchConflictsPhase71python3.13-nixbot> Running phase: pythonRemoveBinBytecodePhase72python3.13-nixbot> Running phase: pythonImportsCheckPhase73python3.13-nixbot> Executing pythonImportsCheckPhase74building '/nix/store/7n7vrbdb75f3sfcylc46himhxj5jfh3c-unit-nixbot.service.drv'75building '/nix/store/rg9wjl11049132lpz4xw91vl8d5d3i9v-system-units.drv'76building '/nix/store/6qdj8rxk0ys9d4y5n6m5zmybp80f23g9-etc.drv'77building '/nix/store/ln8lcgjaq1w4cbhdpv1qxa1d4ikn05ms-activate.drv'78building '/nix/store/dwzwlfnj7q0d1xiwpbcssn8fy7l7s2fi-nixos-system-gitlab-test.drv'79nixos-system-gitlab-test> structuredAttrs is enabled80building '/nix/store/5vavimybd82gfghgx6cvbbnkbqxdiwyn-closure-info.drv'81closure-info> structuredAttrs is enabled82building '/nix/store/h73kxxssslbfjqzr9gcwkyvgy9k1pxkk-closure-info.drv'83closure-info> structuredAttrs is enabled84building '/nix/store/8f7f5bb5njxxr8w4gny8zsna94697cgw-run-nixos-vm.drv'85building '/nix/store/jaw0j84cs3pbzyml6f99j07bxjh2r3lr-nixos-vm.drv'86building '/nix/store/6mxvfzdi3rjwfv0ia1i8xd4yh8s3g6ly-driverConfiguration.json.drv'87driverConfiguration.json> structuredAttrs is enabled88building '/nix/store/94a8j8fzj6kjxxbz7j6sf440rmgqv94h-nixos-test-driver-nixbot-gitlab.drv'89nixos-test-driver-nixbot-gitlab> Running type check (enable/disable: config.skipTypeCheck)90nixos-test-driver-nixbot-gitlab> See https://nixos.org/manual/nixos/stable/#test-opt-skipTypeCheck91nixos-test-driver-nixbot-gitlab> All checks passed!92nixos-test-driver-nixbot-gitlab> Linting test script (enable/disable: config.skipLint)93nixos-test-driver-nixbot-gitlab> See https://nixos.org/manual/nixos/stable/#test-opt-skipLint94nixos-test-driver-nixbot-gitlab> All checks passed!95building '/nix/store/5693hy6nhi7cj4llpx19b2mgi54fysfz-vm-test-run-nixbot-gitlab.drv' on 'ssh-ng://nix@jamie'96building '/nix/store/5693hy6nhi7cj4llpx19b2mgi54fysfz-vm-test-run-nixbot-gitlab.drv'97vm-test-run-nixbot-gitlab> Machine state will be reset. To keep it, pass --keep-machine-state98vm-test-run-nixbot-gitlab> start all VLans99vm-test-run-nixbot-gitlab> (finished: start all VLans, in 0.00 seconds)100vm-test-run-nixbot-gitlab> Test will time out and terminate in 3600 seconds101vm-test-run-nixbot-gitlab> run the VM test script102vm-test-run-nixbot-gitlab> additionally exposed symbols:103vm-test-run-nixbot-gitlab>     gitlab,104vm-test-run-nixbot-gitlab>     vlan1,105vm-test-run-nixbot-gitlab>     start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh106vm-test-run-nixbot-gitlab> start all VMs107vm-test-run-nixbot-gitlab> gitlab: starting vm108vm-test-run-nixbot-gitlab> gitlab: QEMU running (pid 13)109vm-test-run-nixbot-gitlab> gitlab # Disk image does not exist, creating the virtualisation disk image...110vm-test-run-nixbot-gitlab> gitlab # Formatting '/build/vm-state-gitlab/tmp.czATyi2jnL', fmt=raw size=1073741824111vm-test-run-nixbot-gitlab> gitlab # mke2fs 1.47.3 (8-Jul-2025)112vm-test-run-nixbot-gitlab> gitlab # Discarding device blocks:      0/262144             done113vm-test-run-nixbot-gitlab> gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes114vm-test-run-nixbot-gitlab> (finished: start all VMs, in 41.47 seconds)115vm-test-run-nixbot-gitlab> gitlab # Filesystem UUID: 1a2b86b8-d6c6-49de-ae7d-5f550ad9edd2116vm-test-run-nixbot-gitlab> gitlab: waiting for unit setup-gitlab.service117vm-test-run-nixbot-gitlab> gitlab # Superblock backups stored on blocks:118vm-test-run-nixbot-gitlab> gitlab: waiting for the VM to finish booting119vm-test-run-nixbot-gitlab> gitlab # 	32768, 98304, 163840, 229376120vm-test-run-nixbot-gitlab> gitlab # 121vm-test-run-nixbot-gitlab> gitlab # Allocating group tables: 0/8   done122vm-test-run-nixbot-gitlab> gitlab # Writing inode tables: 0/8   done123vm-test-run-nixbot-gitlab> gitlab # Creating journal (8192 blocks): done124vm-test-run-nixbot-gitlab> gitlab # Writing superblocks and filesystem accounting information: 0/8   done125vm-test-run-nixbot-gitlab> gitlab # 126vm-test-run-nixbot-gitlab> gitlab # Virtualisation disk image created.127vm-test-run-nixbot-gitlab> gitlab # Creating Nix store image...128vm-test-run-nixbot-gitlab> gitlab # Created Nix store image.129vm-test-run-nixbot-gitlab> gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)130vm-test-run-nixbot-gitlab> gitlab # 131vm-test-run-nixbot-gitlab> gitlab # 132vm-test-run-nixbot-gitlab> gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA00133vm-test-run-nixbot-gitlab> gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)...134vm-test-run-nixbot-gitlab> gitlab # 135vm-test-run-nixbot-gitlab> gitlab # 136vm-test-run-nixbot-gitlab> gitlab # 137vm-test-run-nixbot-gitlab> gitlab # 138vm-test-run-nixbot-gitlab> gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB00139vm-test-run-nixbot-gitlab> gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)...140vm-test-run-nixbot-gitlab> gitlab # 141vm-test-run-nixbot-gitlab> gitlab # 142vm-test-run-nixbot-gitlab> gitlab # Booting from ROM...143vm-test-run-nixbot-gitlab> gitlab # Probing EDD (edd=off to disable)... ok144vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Linux version 6.18.34 (nixbld@localhost) (gcc (GCC) 15.2.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Jun  1 15:51:08 UTC 2026145vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/d4fjygpaj45mp5xmmj7mz98m76miz2v3-nixos-system-gitlab-test/init regInfo=/nix/store/rsxspnkan5019ra1dc6jhwxvhgcl4vdx-closure-info/registration console=ttyS0,115200n8 console=tty0146vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-provided physical RAM map:147vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable148vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved149vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved150vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable151vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved152vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved153vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved154vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable155vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved156vm-test-run-nixbot-gitlab> gitlab # [    0.000000] NX (Execute Disable) protection: active157vm-test-run-nixbot-gitlab> gitlab # [    0.000000] APIC: Static calls initialized158vm-test-run-nixbot-gitlab> gitlab # [    0.000000] SMBIOS 2.8 present.159vm-test-run-nixbot-gitlab> gitlab # [    0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014160vm-test-run-nixbot-gitlab> gitlab # [    0.000000] DMI: Memory slots populated: 1/1161vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Hypervisor detected: KVM162vm-test-run-nixbot-gitlab> gitlab # [    0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000163vm-test-run-nixbot-gitlab> gitlab # [    0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00164vm-test-run-nixbot-gitlab> gitlab # [    0.000000] kvm-clock: using sched offset of 486732844 cycles165vm-test-run-nixbot-gitlab> gitlab # [    0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns166vm-test-run-nixbot-gitlab> gitlab # [    0.000005] tsc: Detected 2400.008 MHz processor167vm-test-run-nixbot-gitlab> gitlab # [    0.000810] last_pfn = 0x1c0000 max_arch_pfn = 0x10000000000168vm-test-run-nixbot-gitlab> gitlab # [    0.000847] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs169vm-test-run-nixbot-gitlab> gitlab # [    0.000850] x86/PAT: Configuration [0-7]: WB  WC  UC- UC  WB  WP  UC- WT170vm-test-run-nixbot-gitlab> gitlab # [    0.000901] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000171vm-test-run-nixbot-gitlab> gitlab # [    0.002753] found SMP MP-table at [mem 0x000f5470-0x000f547f]172vm-test-run-nixbot-gitlab> gitlab # [    0.002764] Using GB pages for direct mapping173vm-test-run-nixbot-gitlab> gitlab # [    0.002838] RAMDISK: [mem 0xbe4e0000-0xbffcffff]174vm-test-run-nixbot-gitlab> gitlab # [    0.002846] ACPI: Early table checksum verification disabled175vm-test-run-nixbot-gitlab> gitlab # [    0.002848] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )176vm-test-run-nixbot-gitlab> gitlab # [    0.002852] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS  BXPC     00000001 BXPC 00000001)177vm-test-run-nixbot-gitlab> gitlab # [    0.002856] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS  BXPC     00000001 BXPC 00000001)178vm-test-run-nixbot-gitlab> gitlab # [    0.002863] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS  BXPC     00000001 BXPC 00000001)179vm-test-run-nixbot-gitlab> gitlab # [    0.002865] ACPI: FACS 0x00000000BFFE0000 000040180vm-test-run-nixbot-gitlab> gitlab # [    0.002867] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS  BXPC     00000001 BXPC 00000001)181vm-test-run-nixbot-gitlab> gitlab # [    0.002868] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS  BXPC     00000001 BXPC 00000001)182vm-test-run-nixbot-gitlab> gitlab # [    0.002870] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS  BXPC     00000001 BXPC 00000001)183vm-test-run-nixbot-gitlab> gitlab # [    0.002871] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2]184vm-test-run-nixbot-gitlab> gitlab # [    0.002872] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e]185vm-test-run-nixbot-gitlab> gitlab # [    0.002873] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f]186vm-test-run-nixbot-gitlab> gitlab # [    0.002874] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482]187vm-test-run-nixbot-gitlab> gitlab # [    0.002874] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba]188vm-test-run-nixbot-gitlab> gitlab # [    0.002874] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2]189vm-test-run-nixbot-gitlab> gitlab # [    0.003358] No NUMA configuration found190vm-test-run-nixbot-gitlab> gitlab # [    0.003359] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff]191vm-test-run-nixbot-gitlab> gitlab # [    0.003363] NODE_DATA(0) allocated [mem 0x1bfff8780-0x1bfffdcff]192vm-test-run-nixbot-gitlab> gitlab # [    0.005848] Zone ranges:193vm-test-run-nixbot-gitlab> gitlab # [    0.005849]   DMA      [mem 0x0000000000001000-0x0000000000ffffff]194vm-test-run-nixbot-gitlab> gitlab # [    0.005851]   DMA32    [mem 0x0000000001000000-0x00000000ffffffff]195vm-test-run-nixbot-gitlab> gitlab # [    0.005852]   Normal   [mem 0x0000000100000000-0x00000001bfffffff]196vm-test-run-nixbot-gitlab> gitlab # [    0.005853]   Device   empty197vm-test-run-nixbot-gitlab> gitlab # [    0.005853] Movable zone start for each node198vm-test-run-nixbot-gitlab> gitlab # [    0.005854] Early memory node ranges199vm-test-run-nixbot-gitlab> gitlab # [    0.005855]   node   0: [mem 0x0000000000001000-0x000000000009efff]200vm-test-run-nixbot-gitlab> gitlab # [    0.005856]   node   0: [mem 0x0000000000100000-0x00000000bffd7fff]201vm-test-run-nixbot-gitlab> gitlab # [    0.005857]   node   0: [mem 0x0000000100000000-0x00000001bfffffff]202vm-test-run-nixbot-gitlab> gitlab # [    0.005858] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff]203vm-test-run-nixbot-gitlab> gitlab # [    0.005878] On node 0, zone DMA: 1 pages in unavailable ranges204vm-test-run-nixbot-gitlab> gitlab # [    0.006175] On node 0, zone DMA: 97 pages in unavailable ranges205vm-test-run-nixbot-gitlab> gitlab # [    0.120904] On node 0, zone Normal: 40 pages in unavailable ranges206vm-test-run-nixbot-gitlab> gitlab # [    0.121924] ACPI: PM-Timer IO Port: 0x608207vm-test-run-nixbot-gitlab> gitlab # [    0.121942] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])208vm-test-run-nixbot-gitlab> gitlab # [    0.121982] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23209vm-test-run-nixbot-gitlab> gitlab # [    0.121985] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)210vm-test-run-nixbot-gitlab> gitlab # [    0.121987] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)211vm-test-run-nixbot-gitlab> gitlab # [    0.121988] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)212vm-test-run-nixbot-gitlab> gitlab # [    0.121989] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)213vm-test-run-nixbot-gitlab> gitlab # [    0.121990] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)214vm-test-run-nixbot-gitlab> gitlab # [    0.121992] ACPI: Using ACPI (MADT) for SMP configuration information215vm-test-run-nixbot-gitlab> gitlab # [    0.121994] ACPI: HPET id: 0x8086a201 base: 0xfed00000216vm-test-run-nixbot-gitlab> gitlab # [    0.122000] TSC deadline timer available217vm-test-run-nixbot-gitlab> gitlab # [    0.122004] CPU topo: Max. logical packages:   1218vm-test-run-nixbot-gitlab> gitlab # [    0.122005] CPU topo: Max. logical dies:       1219vm-test-run-nixbot-gitlab> gitlab # [    0.122005] CPU topo: Max. dies per package:   1220vm-test-run-nixbot-gitlab> gitlab # [    0.122009] CPU topo: Max. threads per core:   1221vm-test-run-nixbot-gitlab> gitlab # [    0.122009] CPU topo: Num. cores per package:     4222vm-test-run-nixbot-gitlab> gitlab # [    0.122010] CPU topo: Num. threads per package:   4223vm-test-run-nixbot-gitlab> gitlab # [    0.122010] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs224vm-test-run-nixbot-gitlab> gitlab # [    0.122035] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()225vm-test-run-nixbot-gitlab> gitlab # [    0.122052] kvm-guest: KVM setup pv remote TLB flush226vm-test-run-nixbot-gitlab> gitlab # [    0.122056] kvm-guest: setup PV sched yield227vm-test-run-nixbot-gitlab> gitlab # [    0.122079] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]228vm-test-run-nixbot-gitlab> gitlab # [    0.122081] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]229vm-test-run-nixbot-gitlab> gitlab # [    0.122082] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff]230vm-test-run-nixbot-gitlab> gitlab # [    0.122083] [mem 0xc0000000-0xfeffbfff] available for PCI devices231vm-test-run-nixbot-gitlab> gitlab # [    0.122085] Booting paravirtualized kernel on KVM232vm-test-run-nixbot-gitlab> gitlab # [    0.122088] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns233vm-test-run-nixbot-gitlab> gitlab # [    0.126537] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1234vm-test-run-nixbot-gitlab> gitlab # [    0.129050] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288235vm-test-run-nixbot-gitlab> gitlab # [    0.129109] kvm-guest: PV spinlocks enabled236vm-test-run-nixbot-gitlab> gitlab # [    0.129111] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear)237vm-test-run-nixbot-gitlab> gitlab # [    0.129114] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/d4fjygpaj45mp5xmmj7mz98m76miz2v3-nixos-system-gitlab-test/init regInfo=/nix/store/rsxspnkan5019ra1dc6jhwxvhgcl4vdx-closure-info/registration console=ttyS0,115200n8 console=tty0238vm-test-run-nixbot-gitlab> gitlab # [    0.129209] Unknown kernel command line parameters "regInfo=/nix/store/rsxspnkan5019ra1dc6jhwxvhgcl4vdx-closure-info/registration", will be passed to user space.239vm-test-run-nixbot-gitlab> gitlab # [    0.129222] random: crng init done240vm-test-run-nixbot-gitlab> gitlab # [    0.129222] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes241vm-test-run-nixbot-gitlab> gitlab # [    0.138309] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)242vm-test-run-nixbot-gitlab> gitlab # [    0.142725] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)243vm-test-run-nixbot-gitlab> gitlab # [    0.142800] software IO TLB: area num 4.244vm-test-run-nixbot-gitlab> gitlab # [    0.218803] Fallback order for Node 0: 0245vm-test-run-nixbot-gitlab> gitlab # [    0.218811] Built 1 zonelists, mobility grouping on.  Total pages: 1572726246vm-test-run-nixbot-gitlab> gitlab # [    0.218813] Policy zone: Normal247vm-test-run-nixbot-gitlab> gitlab # [    0.221650] mem auto-init: stack:all(zero), heap alloc:on, heap free:off248vm-test-run-nixbot-gitlab> gitlab # [    0.231011] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1249vm-test-run-nixbot-gitlab> gitlab # [    0.245042] allocated 12582912 bytes of page_ext250vm-test-run-nixbot-gitlab> gitlab # [    0.255096] ftrace: allocating 48565 entries in 192 pages251vm-test-run-nixbot-gitlab> gitlab # [    0.255103] ftrace: allocated 192 pages with 2 groups252vm-test-run-nixbot-gitlab> gitlab # [    0.255973] Dynamic Preempt: lazy253vm-test-run-nixbot-gitlab> gitlab # [    0.256172] rcu: Preemptible hierarchical RCU implementation.254vm-test-run-nixbot-gitlab> gitlab # [    0.256173] rcu: 	RCU event tracing is enabled.255vm-test-run-nixbot-gitlab> gitlab # [    0.256174] rcu: 	RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.256vm-test-run-nixbot-gitlab> gitlab # [    0.256176] 	Trampoline variant of Tasks RCU enabled.257vm-test-run-nixbot-gitlab> gitlab # [    0.256176] 	Rude variant of Tasks RCU enabled.258vm-test-run-nixbot-gitlab> gitlab # [    0.256176] 	Tracing variant of Tasks RCU enabled.259vm-test-run-nixbot-gitlab> gitlab # [    0.256177] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.260vm-test-run-nixbot-gitlab> gitlab # [    0.256178] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4261vm-test-run-nixbot-gitlab> gitlab # [    0.256192] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.262vm-test-run-nixbot-gitlab> gitlab # [    0.256194] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.263vm-test-run-nixbot-gitlab> gitlab # [    0.256195] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.264vm-test-run-nixbot-gitlab> gitlab # [    0.260570] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16265vm-test-run-nixbot-gitlab> gitlab # [    0.260872] rcu: srcu_init: Setting srcu_struct sizes based on contention.266vm-test-run-nixbot-gitlab> gitlab # [    0.260988] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)267vm-test-run-nixbot-gitlab> gitlab # [    0.268173] Console: colour VGA+ 80x25268vm-test-run-nixbot-gitlab> gitlab # [    0.268176] printk: legacy console [tty0] enabled269vm-test-run-nixbot-gitlab> gitlab # [    0.311423] printk: legacy console [ttyS0] enabled270vm-test-run-nixbot-gitlab> gitlab # [    0.509494] ACPI: Core revision 20250807271vm-test-run-nixbot-gitlab> gitlab # [    0.511055] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns272vm-test-run-nixbot-gitlab> gitlab # [    0.513896] APIC: Switch to symmetric I/O mode setup273vm-test-run-nixbot-gitlab> gitlab # [    0.515639] x2apic enabled274vm-test-run-nixbot-gitlab> gitlab # [    0.516877] APIC: Switched APIC routing to: physical x2apic275vm-test-run-nixbot-gitlab> gitlab # [    0.518583] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask()276vm-test-run-nixbot-gitlab> gitlab # [    0.520810] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself()277vm-test-run-nixbot-gitlab> gitlab # [    0.523575] kvm-guest: setup PV IPIs278vm-test-run-nixbot-gitlab> gitlab # [    0.525856] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1279vm-test-run-nixbot-gitlab> gitlab # [    0.527702] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x22983f10a64, max_idle_ns: 440795218721 ns280vm-test-run-nixbot-gitlab> gitlab # [    0.530765] Calibrating delay loop (skipped) preset value.. 4800.01 BogoMIPS (lpj=2400008)281vm-test-run-nixbot-gitlab> gitlab # [    0.532883] x86/cpu: User Mode Instruction Prevention (UMIP) activated282vm-test-run-nixbot-gitlab> gitlab # [    0.533911] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127283vm-test-run-nixbot-gitlab> gitlab # [    0.534763] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0284vm-test-run-nixbot-gitlab> gitlab # [    0.535769] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto285vm-test-run-nixbot-gitlab> gitlab # [    0.537763] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl286vm-test-run-nixbot-gitlab> gitlab # [    0.539764] Spectre V2 : Mitigation: Enhanced / Automatic IBRS287vm-test-run-nixbot-gitlab> gitlab # [    0.540763] Speculative Return Stack Overflow: Mitigation: Safe RET288vm-test-run-nixbot-gitlab> gitlab # [    0.541762] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization289vm-test-run-nixbot-gitlab> gitlab # [    0.542770] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier290vm-test-run-nixbot-gitlab> gitlab # [    0.544763] active return thunk: srso_alias_return_thunk291vm-test-run-nixbot-gitlab> gitlab # [    0.545795] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'292vm-test-run-nixbot-gitlab> gitlab # [    0.547763] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'293vm-test-run-nixbot-gitlab> gitlab # [    0.548763] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'294vm-test-run-nixbot-gitlab> gitlab # [    0.550763] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'295vm-test-run-nixbot-gitlab> gitlab # [    0.552763] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'296vm-test-run-nixbot-gitlab> gitlab # [    0.554763] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'297vm-test-run-nixbot-gitlab> gitlab # [    0.555763] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'298vm-test-run-nixbot-gitlab> gitlab # [    0.556764] x86/fpu: Supporting XSAVE feature 0x800: 'Control-flow User registers'299vm-test-run-nixbot-gitlab> gitlab # [    0.558763] x86/fpu: Supporting XSAVE feature 0x1000: 'Control-flow Kernel registers (KVM only)'300vm-test-run-nixbot-gitlab> gitlab # [    0.559763] x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256301vm-test-run-nixbot-gitlab> gitlab # [    0.561763] x86/fpu: xstate_offset[5]:  832, xstate_sizes[5]:   64302vm-test-run-nixbot-gitlab> gitlab # [    0.562763] x86/fpu: xstate_offset[6]:  896, xstate_sizes[6]:  512303vm-test-run-nixbot-gitlab> gitlab # [    0.563763] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024304vm-test-run-nixbot-gitlab> gitlab # [    0.564762] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]:    8305vm-test-run-nixbot-gitlab> gitlab # [    0.565763] x86/fpu: xstate_offset[11]: 2440, xstate_sizes[11]:   16306vm-test-run-nixbot-gitlab> gitlab # [    0.567762] x86/fpu: xstate_offset[12]: 2456, xstate_sizes[12]:   24307vm-test-run-nixbot-gitlab> gitlab # [    0.569762] x86/fpu: Enabled xstate features 0x1ae7, context size is 2480 bytes, using 'compacted' format.308vm-test-run-nixbot-gitlab> gitlab # [    0.600367] Freeing SMP alternatives memory: 44K309vm-test-run-nixbot-gitlab> gitlab # [    0.600765] pid_max: default: 32768 minimum: 301310vm-test-run-nixbot-gitlab> gitlab # [    0.601888] LSM: initializing lsm=capability,landlock,yama,bpf,ima311vm-test-run-nixbot-gitlab> gitlab # [    0.602879] landlock: Up and running.312vm-test-run-nixbot-gitlab> gitlab # [    0.603763] Yama: becoming mindful.313vm-test-run-nixbot-gitlab> gitlab # [    0.605916] LSM support for eBPF active314vm-test-run-nixbot-gitlab> gitlab # [    0.606992] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)315vm-test-run-nixbot-gitlab> gitlab # [    0.609818] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)316vm-test-run-nixbot-gitlab> gitlab # [    0.612446] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)317vm-test-run-nixbot-gitlab> gitlab # [    0.613359] Performance Events: Fam17h+ core perfctr, AMD PMU driver.318vm-test-run-nixbot-gitlab> gitlab # [    0.613768] ... version:                   2319vm-test-run-nixbot-gitlab> gitlab # [    0.614765] ... bit width:                 48320vm-test-run-nixbot-gitlab> gitlab # [    0.615825] ... generic counters:          6321vm-test-run-nixbot-gitlab> gitlab # [    0.616765] ... generic bitmap:            000000000000003f322vm-test-run-nixbot-gitlab> gitlab # [    0.617765] ... fixed-purpose counters:    0323vm-test-run-nixbot-gitlab> gitlab # [    0.618765] ... fixed-purpose bitmap:      0000000000000000324vm-test-run-nixbot-gitlab> gitlab # [    0.619765] ... value mask:                0000ffffffffffff325vm-test-run-nixbot-gitlab> gitlab # [    0.620765] ... max period:                00007fffffffffff326vm-test-run-nixbot-gitlab> gitlab # [    0.621765] ... global_ctrl mask:          000000000000003f327vm-test-run-nixbot-gitlab> gitlab # [    0.622889] signal: max sigframe size: 3376328vm-test-run-nixbot-gitlab> gitlab # [    0.623865] rcu: Hierarchical SRCU implementation.329vm-test-run-nixbot-gitlab> gitlab # [    0.624769] rcu: 	Max phase no-delay instances is 400.330vm-test-run-nixbot-gitlab> gitlab # [    0.625979] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level331vm-test-run-nixbot-gitlab> gitlab # [    0.631378] smp: Bringing up secondary CPUs ...332vm-test-run-nixbot-gitlab> gitlab # [    0.632178] smpboot: x86: Booting SMP configuration:333vm-test-run-nixbot-gitlab> gitlab # [    0.632777] .... node  #0, CPUs:      #1 #2 #3334vm-test-run-nixbot-gitlab> gitlab # [    0.636857] smp: Brought up 1 node, 4 CPUs335vm-test-run-nixbot-gitlab> gitlab # [    0.638764] smpboot: Total of 4 processors activated (19200.06 BogoMIPS)336vm-test-run-nixbot-gitlab> gitlab # [    0.640416] Memory: 6013416K/6290904K available (17098K kernel code, 2721K rwdata, 13520K rodata, 3644K init, 3008K bss, 258300K reserved, 0K cma-reserved)337vm-test-run-nixbot-gitlab> gitlab # [    0.642763] devtmpfs: initialized338vm-test-run-nixbot-gitlab> gitlab # [    0.644001] x86/mm: Memory block size: 128MB339vm-test-run-nixbot-gitlab> gitlab # [    0.646918] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns340vm-test-run-nixbot-gitlab> gitlab # [    0.648817] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)341vm-test-run-nixbot-gitlab> gitlab # [    0.649851] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).342vm-test-run-nixbot-gitlab> gitlab # [    0.650913] pinctrl core: initialized pinctrl subsystem343vm-test-run-nixbot-gitlab> gitlab # [    0.652049] PM: RTC time: 14:58:59, date: 2026-06-14344vm-test-run-nixbot-gitlab> gitlab # [    0.655819] NET: Registered PF_NETLINK/PF_ROUTE protocol family345vm-test-run-nixbot-gitlab> gitlab # [    0.658159] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations346vm-test-run-nixbot-gitlab> gitlab # [    0.658826] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations347vm-test-run-nixbot-gitlab> gitlab # [    0.660894] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations348vm-test-run-nixbot-gitlab> gitlab # [    0.661779] audit: initializing netlink subsys (disabled)349vm-test-run-nixbot-gitlab> gitlab # [    0.662942] audit: type=2000 audit(1781449139.906:1): state=initialized audit_enabled=0 res=1350vm-test-run-nixbot-gitlab> gitlab # [    0.663091] thermal_sys: Registered thermal governor 'fair_share'351vm-test-run-nixbot-gitlab> gitlab # [    0.664765] thermal_sys: Registered thermal governor 'bang_bang'352vm-test-run-nixbot-gitlab> gitlab # [    0.665766] thermal_sys: Registered thermal governor 'step_wise'353vm-test-run-nixbot-gitlab> gitlab # [    0.666765] thermal_sys: Registered thermal governor 'user_space'354vm-test-run-nixbot-gitlab> gitlab # [    0.667765] thermal_sys: Registered thermal governor 'power_allocator'355vm-test-run-nixbot-gitlab> gitlab # [    0.668784] cpuidle: using governor menu356vm-test-run-nixbot-gitlab> gitlab # [    0.671992] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5357vm-test-run-nixbot-gitlab> gitlab # [    0.673089] PCI: Using configuration type 1 for base access358vm-test-run-nixbot-gitlab> gitlab # [    0.673766] PCI: Using configuration type 1 for extended access359vm-test-run-nixbot-gitlab> gitlab # [    0.674902] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.360vm-test-run-nixbot-gitlab> gitlab # [    0.677808] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages361vm-test-run-nixbot-gitlab> gitlab # [    0.678765] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page362vm-test-run-nixbot-gitlab> gitlab # [    0.679766] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages363vm-test-run-nixbot-gitlab> gitlab # [    0.680765] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page364vm-test-run-nixbot-gitlab> gitlab # [    0.686147] ACPI: Added _OSI(Module Device)365vm-test-run-nixbot-gitlab> gitlab # [    0.688772] ACPI: Added _OSI(Processor Device)366vm-test-run-nixbot-gitlab> gitlab # [    0.692776] ACPI: Added _OSI(Processor Aggregator Device)367vm-test-run-nixbot-gitlab> gitlab # [    0.701800] ACPI: 1 ACPI AML tables successfully acquired and loaded368vm-test-run-nixbot-gitlab> gitlab # [    0.711292] ACPI: Interpreter enabled369vm-test-run-nixbot-gitlab> gitlab # [    0.713917] ACPI: PM: (supports S0 S3 S4 S5)370vm-test-run-nixbot-gitlab> gitlab # [    0.717779] ACPI: Using IOAPIC for interrupt routing371vm-test-run-nixbot-gitlab> gitlab # [    0.721090] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug372vm-test-run-nixbot-gitlab> gitlab # [    0.727772] PCI: Using E820 reservations for host bridge windows373vm-test-run-nixbot-gitlab> gitlab # [    0.733340] ACPI: Enabled 2 GPEs in block 00 to 0F374vm-test-run-nixbot-gitlab> gitlab # [    0.748820] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])375vm-test-run-nixbot-gitlab> gitlab # [    0.752781] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]376vm-test-run-nixbot-gitlab> gitlab # [    0.759099] acpiphp: Slot [3] registered377vm-test-run-nixbot-gitlab> gitlab # [    0.761917] acpiphp: Slot [4] registered378vm-test-run-nixbot-gitlab> gitlab # [    0.763918] acpiphp: Slot [5] registered379vm-test-run-nixbot-gitlab> gitlab # [    0.766937] acpiphp: Slot [6] registered380vm-test-run-nixbot-gitlab> gitlab # [    0.768825] acpiphp: Slot [7] registered381vm-test-run-nixbot-gitlab> gitlab # [    0.769832] acpiphp: Slot [8] registered382vm-test-run-nixbot-gitlab> gitlab # [    0.771834] acpiphp: Slot [9] registered383vm-test-run-nixbot-gitlab> gitlab # [    0.772853] acpiphp: Slot [10] registered384vm-test-run-nixbot-gitlab> gitlab # [    0.773839] acpiphp: Slot [11] registered385vm-test-run-nixbot-gitlab> gitlab # [    0.775833] acpiphp: Slot [12] registered386vm-test-run-nixbot-gitlab> gitlab # [    0.776839] acpiphp: Slot [13] registered387vm-test-run-nixbot-gitlab> gitlab # [    0.778042] acpiphp: Slot [14] registered388vm-test-run-nixbot-gitlab> gitlab # [    0.781908] acpiphp: Slot [15] registered389vm-test-run-nixbot-gitlab> gitlab # [    0.784861] acpiphp: Slot [16] registered390vm-test-run-nixbot-gitlab> gitlab # [    0.787870] acpiphp: Slot [17] registered391vm-test-run-nixbot-gitlab> gitlab # [    0.791865] acpiphp: Slot [18] registered392vm-test-run-nixbot-gitlab> gitlab # [    0.794914] acpiphp: Slot [19] registered393vm-test-run-nixbot-gitlab> gitlab # [    0.797876] acpiphp: Slot [20] registered394vm-test-run-nixbot-gitlab> gitlab # [    0.800863] acpiphp: Slot [21] registered395vm-test-run-nixbot-gitlab> gitlab # [    0.804919] acpiphp: Slot [22] registered396vm-test-run-nixbot-gitlab> gitlab # [    0.807863] acpiphp: Slot [23] registered397vm-test-run-nixbot-gitlab> gitlab # [    0.810884] acpiphp: Slot [24] registered398vm-test-run-nixbot-gitlab> gitlab # [    0.813867] acpiphp: Slot [25] registered399vm-test-run-nixbot-gitlab> gitlab # [    0.817918] acpiphp: Slot [26] registered400vm-test-run-nixbot-gitlab> gitlab # [    0.820865] acpiphp: Slot [27] registered401vm-test-run-nixbot-gitlab> gitlab # [    0.823964] acpiphp: Slot [28] registered402vm-test-run-nixbot-gitlab> gitlab # [    0.826840] acpiphp: Slot [29] registered403vm-test-run-nixbot-gitlab> gitlab # [    0.829902] acpiphp: Slot [30] registered404vm-test-run-nixbot-gitlab> gitlab # [    0.832840] acpiphp: Slot [31] registered405vm-test-run-nixbot-gitlab> gitlab # [    0.834847] PCI host bridge to bus 0000:00406vm-test-run-nixbot-gitlab> gitlab # [    0.837905] pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7 window]407vm-test-run-nixbot-gitlab> gitlab # [    0.841778] pci_bus 0000:00: root bus resource [io  0x0d00-0xffff window]408vm-test-run-nixbot-gitlab> gitlab # [    0.845787] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]409vm-test-run-nixbot-gitlab> gitlab # [    0.847767] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]410vm-test-run-nixbot-gitlab> gitlab # [    0.851774] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window]411vm-test-run-nixbot-gitlab> gitlab # [    0.857827] pci_bus 0000:00: root bus resource [bus 00-ff]412vm-test-run-nixbot-gitlab> gitlab # [    0.863153] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint413vm-test-run-nixbot-gitlab> gitlab # [    0.872445] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint414vm-test-run-nixbot-gitlab> gitlab # [    0.881159] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint415vm-test-run-nixbot-gitlab> gitlab # [    0.891923] pci 0000:00:01.1: BAR 4 [io  0xc220-0xc22f]416vm-test-run-nixbot-gitlab> gitlab # [    0.894902] pci 0000:00:01.1: BAR 0 [io  0x01f0-0x01f7]: legacy IDE quirk417vm-test-run-nixbot-gitlab> gitlab # [    0.898771] pci 0000:00:01.1: BAR 1 [io  0x03f6]: legacy IDE quirk418vm-test-run-nixbot-gitlab> gitlab # [    0.902771] pci 0000:00:01.1: BAR 2 [io  0x0170-0x0177]: legacy IDE quirk419vm-test-run-nixbot-gitlab> gitlab # [    0.906776] pci 0000:00:01.1: BAR 3 [io  0x0376]: legacy IDE quirk420vm-test-run-nixbot-gitlab> gitlab # [    0.911111] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint421vm-test-run-nixbot-gitlab> gitlab # [    0.915933] pci 0000:00:01.2: BAR 4 [io  0xc140-0xc15f]422vm-test-run-nixbot-gitlab> gitlab # [    0.923395] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint423vm-test-run-nixbot-gitlab> gitlab # [    0.931512] pci 0000:00:01.3: quirk: [io  0x0600-0x063f] claimed by PIIX4 ACPI424vm-test-run-nixbot-gitlab> gitlab # [    0.936808] pci 0000:00:01.3: quirk: [io  0x0700-0x070f] claimed by PIIX4 SMB425vm-test-run-nixbot-gitlab> gitlab # [    0.941951] pci 0000:00:01.3: quirk_piix4_acpi+0x0/0x180 took 10742 usecs426vm-test-run-nixbot-gitlab> gitlab # [    0.948663] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint427vm-test-run-nixbot-gitlab> gitlab # [    0.967914] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]428vm-test-run-nixbot-gitlab> gitlab # [    0.971824] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]429vm-test-run-nixbot-gitlab> gitlab # [    0.975842] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref]430vm-test-run-nixbot-gitlab> gitlab # [    0.978029] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]431vm-test-run-nixbot-gitlab> gitlab # [    0.992818] pci 0000:00:02.0: pci_fixup_video+0x0/0x100 took 14648 usecs432vm-test-run-nixbot-gitlab> gitlab # [    0.997089] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint433vm-test-run-nixbot-gitlab> gitlab # [    1.012809] pci 0000:00:03.0: BAR 0 [io  0xc160-0xc17f]434vm-test-run-nixbot-gitlab> gitlab # [    1.017807] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]435vm-test-run-nixbot-gitlab> gitlab # [    1.021916] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref]436vm-test-run-nixbot-gitlab> gitlab # [    1.027809] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]437vm-test-run-nixbot-gitlab> gitlab # [    1.038997] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint438vm-test-run-nixbot-gitlab> gitlab # [    1.050801] pci 0000:00:04.0: BAR 0 [io  0xc180-0xc19f]439vm-test-run-nixbot-gitlab> gitlab # [    1.053804] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]440vm-test-run-nixbot-gitlab> gitlab # [    1.058829] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref]441vm-test-run-nixbot-gitlab> gitlab # [    1.062998] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint442vm-test-run-nixbot-gitlab> gitlab # [    1.068804] pci 0000:00:05.0: BAR 0 [io  0xc1a0-0xc1bf]443vm-test-run-nixbot-gitlab> gitlab # [    1.070779] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]444vm-test-run-nixbot-gitlab> gitlab # [    1.071817] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref]445vm-test-run-nixbot-gitlab> gitlab # [    1.076804] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint446vm-test-run-nixbot-gitlab> gitlab # [    1.082841] pci 0000:00:06.0: BAR 0 [io  0xc1c0-0xc1df]447vm-test-run-nixbot-gitlab> gitlab # [    1.083779] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]448vm-test-run-nixbot-gitlab> gitlab # [    1.085818] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref]449vm-test-run-nixbot-gitlab> gitlab # [    1.090816] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint450vm-test-run-nixbot-gitlab> gitlab # [    1.095804] pci 0000:00:07.0: BAR 0 [io  0xc000-0xc07f]451vm-test-run-nixbot-gitlab> gitlab # [    1.097780] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]452vm-test-run-nixbot-gitlab> gitlab # [    1.100819] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref]453vm-test-run-nixbot-gitlab> gitlab # [    1.104816] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint454vm-test-run-nixbot-gitlab> gitlab # [    1.109779] pci 0000:00:08.0: BAR 0 [io  0xc080-0xc0ff]455vm-test-run-nixbot-gitlab> gitlab # [    1.111780] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]456vm-test-run-nixbot-gitlab> gitlab # [    1.113819] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref]457vm-test-run-nixbot-gitlab> gitlab # [    1.117856] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint458vm-test-run-nixbot-gitlab> gitlab # [    1.122804] pci 0000:00:09.0: BAR 0 [io  0xc1e0-0xc1ff]459vm-test-run-nixbot-gitlab> gitlab # [    1.124801] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]460vm-test-run-nixbot-gitlab> gitlab # [    1.129906] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref]461vm-test-run-nixbot-gitlab> gitlab # [    1.134836] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref]462vm-test-run-nixbot-gitlab> gitlab # [    1.145639] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint463vm-test-run-nixbot-gitlab> gitlab # [    1.158840] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]464vm-test-run-nixbot-gitlab> gitlab # [    1.162908] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref]465vm-test-run-nixbot-gitlab> gitlab # [    1.174133] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint466vm-test-run-nixbot-gitlab> gitlab # [    1.187831] pci 0000:00:0b.0: BAR 0 [io  0xc100-0xc13f]467vm-test-run-nixbot-gitlab> gitlab # [    1.190822] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]468vm-test-run-nixbot-gitlab> gitlab # [    1.194925] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref]469vm-test-run-nixbot-gitlab> gitlab # [    1.206160] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint470vm-test-run-nixbot-gitlab> gitlab # [    1.218807] pci 0000:00:0c.0: BAR 0 [io  0xc200-0xc21f]471vm-test-run-nixbot-gitlab> gitlab # [    1.222808] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff]472vm-test-run-nixbot-gitlab> gitlab # [    1.227914] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref]473vm-test-run-nixbot-gitlab> gitlab # [    1.250962] ACPI: PCI: Interrupt link LNKA configured for IRQ 10474vm-test-run-nixbot-gitlab> gitlab # [    1.255276] ACPI: PCI: Interrupt link LNKB configured for IRQ 10475vm-test-run-nixbot-gitlab> gitlab # [    1.259010] ACPI: PCI: Interrupt link LNKC configured for IRQ 11476vm-test-run-nixbot-gitlab> gitlab # [    1.260976] ACPI: PCI: Interrupt link LNKD configured for IRQ 11477vm-test-run-nixbot-gitlab> gitlab # [    1.262877] ACPI: PCI: Interrupt link LNKS configured for IRQ 9478vm-test-run-nixbot-gitlab> gitlab # [    1.266805] iommu: Default domain type: Translated479vm-test-run-nixbot-gitlab> gitlab # [    1.267764] iommu: DMA domain TLB invalidation policy: lazy mode480vm-test-run-nixbot-gitlab> gitlab # [    1.270047] ACPI: bus type USB registered481vm-test-run-nixbot-gitlab> gitlab # [    1.271839] usbcore: registered new interface driver usbfs482vm-test-run-nixbot-gitlab> gitlab # [    1.273785] usbcore: registered new interface driver hub483vm-test-run-nixbot-gitlab> gitlab # [    1.274776] usbcore: registered new device driver usb484vm-test-run-nixbot-gitlab> gitlab # [    1.278170] NetLabel: Initializing485vm-test-run-nixbot-gitlab> gitlab # [    1.278764] NetLabel:  domain hash size = 128486vm-test-run-nixbot-gitlab> gitlab # [    1.280765] NetLabel:  protocols = UNLABELED CIPSOv4 CALIPSO487vm-test-run-nixbot-gitlab> gitlab # [    1.281836] NetLabel:  unlabeled traffic allowed by default488vm-test-run-nixbot-gitlab> gitlab # [    1.283779] PCI: Using ACPI for IRQ routing489vm-test-run-nixbot-gitlab> gitlab # [    1.286073] pci 0000:00:02.0: vgaarb: setting as boot VGA device490vm-test-run-nixbot-gitlab> gitlab # [    1.286759] pci 0000:00:02.0: vgaarb: bridge control possible491vm-test-run-nixbot-gitlab> gitlab # [    1.286759] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none492vm-test-run-nixbot-gitlab> gitlab # [    1.291768] vgaarb: loaded493vm-test-run-nixbot-gitlab> gitlab # [    1.292916] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0494vm-test-run-nixbot-gitlab> gitlab # [    1.294764] hpet0: 3 comparators, 64-bit 100.000000 MHz counter495vm-test-run-nixbot-gitlab> gitlab # [    1.300847] clocksource: Switched to clocksource kvm-clock496vm-test-run-nixbot-gitlab> gitlab # [    1.303293] VFS: Disk quotas dquot_6.6.0497vm-test-run-nixbot-gitlab> gitlab # [    1.304572] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)498vm-test-run-nixbot-gitlab> gitlab # [    1.307005] pnp: PnP ACPI init499vm-test-run-nixbot-gitlab> gitlab # [    1.309441] pnp: PnP ACPI: found 6 devices500vm-test-run-nixbot-gitlab> gitlab # [    1.325047] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns501vm-test-run-nixbot-gitlab> gitlab # [    1.331662] clocksource: Switched to clocksource acpi_pm502vm-test-run-nixbot-gitlab> gitlab # [    1.336338] NET: Registered PF_INET protocol family503vm-test-run-nixbot-gitlab> gitlab # [    1.343521] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)504vm-test-run-nixbot-gitlab> gitlab # [    1.371212] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)505vm-test-run-nixbot-gitlab> gitlab # [    1.373810] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)506vm-test-run-nixbot-gitlab> gitlab # [    1.376261] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)507vm-test-run-nixbot-gitlab> gitlab # [    1.381573] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)508vm-test-run-nixbot-gitlab> gitlab # [    1.383982] TCP: Hash tables configured (established 65536 bind 65536)509vm-test-run-nixbot-gitlab> gitlab # [    1.386143] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)510vm-test-run-nixbot-gitlab> gitlab # [    1.388449] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)511vm-test-run-nixbot-gitlab> gitlab # [    1.390514] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)512vm-test-run-nixbot-gitlab> gitlab # [    1.392868] NET: Registered PF_UNIX/PF_LOCAL protocol family513vm-test-run-nixbot-gitlab> gitlab # [    1.394692] NET: Registered PF_XDP protocol family514vm-test-run-nixbot-gitlab> gitlab # [    1.396265] pci_bus 0000:00: resource 4 [io  0x0000-0x0cf7 window]515vm-test-run-nixbot-gitlab> gitlab # [    1.398110] pci_bus 0000:00: resource 5 [io  0x0d00-0xffff window]516vm-test-run-nixbot-gitlab> gitlab # [    1.399962] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]517vm-test-run-nixbot-gitlab> gitlab # [    1.402005] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window]518vm-test-run-nixbot-gitlab> gitlab # [    1.404023] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window]519vm-test-run-nixbot-gitlab> gitlab # [    1.406352] pci 0000:00:01.0: PIIX3: Enabling Passive Release520vm-test-run-nixbot-gitlab> gitlab # [    1.408136] pci 0000:00:00.0: Limiting direct PCI/PCI transfers521vm-test-run-nixbot-gitlab> gitlab # [    1.411500] ACPI: \_SB_.LNKD: Enabled at IRQ 11522vm-test-run-nixbot-gitlab> gitlab # [    1.415024] PCI: CLS 0 bytes, default 64523vm-test-run-nixbot-gitlab> gitlab # [    1.416477] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)524vm-test-run-nixbot-gitlab> gitlab # [    1.417111] Trying to unpack rootfs image as initramfs...525vm-test-run-nixbot-gitlab> gitlab # [    1.418418] software IO TLB: mapped [mem 0x00000000ba4e0000-0x00000000be4e0000] (64MB)526vm-test-run-nixbot-gitlab> gitlab # [    1.418558] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x22983f10a64, max_idle_ns: 440795218721 ns527vm-test-run-nixbot-gitlab> gitlab # [    1.450206] Initialise system trusted keyrings528vm-test-run-nixbot-gitlab> gitlab # [    1.454537] workingset: timestamp_bits=40 max_order=21 bucket_order=0529vm-test-run-nixbot-gitlab> gitlab # [    1.487450] Key type asymmetric registered530vm-test-run-nixbot-gitlab> gitlab # [    1.490205] Asymmetric key parser 'x509' registered531vm-test-run-nixbot-gitlab> gitlab # [    1.493378] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)532vm-test-run-nixbot-gitlab> gitlab # [    1.499499] io scheduler mq-deadline registered533vm-test-run-nixbot-gitlab> gitlab # [    1.502637] io scheduler kyber registered534vm-test-run-nixbot-gitlab> gitlab # [    1.511210] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled535vm-test-run-nixbot-gitlab> gitlab # [    1.517094] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A536vm-test-run-nixbot-gitlab> gitlab # [    1.527020] Linux agpgart interface v0.103537vm-test-run-nixbot-gitlab> gitlab # [    1.530459] ACPI: bus type drm_connector registered538vm-test-run-nixbot-gitlab> gitlab # [    1.537393] usbcore: registered new interface driver usbserial_generic539vm-test-run-nixbot-gitlab> gitlab # [    1.542501] usbserial: USB Serial support registered for generic540vm-test-run-nixbot-gitlab> gitlab # [    1.547131] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.541vm-test-run-nixbot-gitlab> gitlab # [    1.547131] Please enable it if your BIOS has the CPPC option.542vm-test-run-nixbot-gitlab> gitlab # [    1.557207] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled543vm-test-run-nixbot-gitlab> gitlab # [    1.563306] drop_monitor: Initializing network drop monitor service544vm-test-run-nixbot-gitlab> gitlab # [    1.568501] NET: Registered PF_INET6 protocol family545vm-test-run-nixbot-gitlab> gitlab # [    1.574697] Segment Routing with IPv6546vm-test-run-nixbot-gitlab> gitlab # [    1.577863] In-situ OAM (IOAM) with IPv6547vm-test-run-nixbot-gitlab> gitlab # [    1.584227] IPI shorthand broadcast: enabled548vm-test-run-nixbot-gitlab> gitlab # [    1.598134] sched_clock: Marking stable (1340075466, 257878061)->(1677581358, -79627831)549vm-test-run-nixbot-gitlab> gitlab # [    1.605122] registered taskstats version 1550vm-test-run-nixbot-gitlab> gitlab # [    1.609406] Loading compiled-in X.509 certificates551vm-test-run-nixbot-gitlab> gitlab # [    1.637615] Demotion targets for Node 0: null552vm-test-run-nixbot-gitlab> gitlab # [    1.641313] Key type .fscrypt registered553vm-test-run-nixbot-gitlab> gitlab # [    1.644547] Key type fscrypt-provisioning registered554vm-test-run-nixbot-gitlab> gitlab # [    1.648825] ima: No TPM chip found, activating TPM-bypass!555vm-test-run-nixbot-gitlab> gitlab # [    1.653073] ima: Allocated hash algorithm: sha1556vm-test-run-nixbot-gitlab> gitlab # [    1.656756] ima: No architecture policies found557vm-test-run-nixbot-gitlab> gitlab # [    1.661049] PM:   Magic number: 14:977:990558vm-test-run-nixbot-gitlab> gitlab # [    1.664353] machinecheck machinecheck3: hash matches559vm-test-run-nixbot-gitlab> gitlab # [    1.668390] memory memory33: hash matches560vm-test-run-nixbot-gitlab> gitlab # [    1.673731] RAS: Correctable Errors collector initialized.561vm-test-run-nixbot-gitlab> gitlab # [    1.685440] clk: Disabling unused clocks562vm-test-run-nixbot-gitlab> gitlab # [    1.688007] PM: genpd: Disabling unused power domains563vm-test-run-nixbot-gitlab> gitlab # [    1.701675] Freeing initrd memory: 27584K564vm-test-run-nixbot-gitlab> gitlab # [    1.708563] Freeing unused decrypted memory: 2028K565vm-test-run-nixbot-gitlab> gitlab # [    1.715559] Freeing unused kernel image (initmem) memory: 3644K566vm-test-run-nixbot-gitlab> gitlab # [    1.719381] Write protecting the kernel read-only data: 32768k567vm-test-run-nixbot-gitlab> gitlab # [    1.725370] Freeing unused kernel image (text/rodata gap) memory: 1332K568vm-test-run-nixbot-gitlab> gitlab # [    1.728077] Freeing unused kernel image (rodata/data gap) memory: 816K569vm-test-run-nixbot-gitlab> gitlab # [    1.780405] x86/mm: Checked W+X mappings: passed, no W+X pages found.570vm-test-run-nixbot-gitlab> gitlab # [    1.782404] Run /init as init process571vm-test-run-nixbot-gitlab> gitlab # [    1.798301] systemd[1]: Inserted module 'autofs4'572vm-test-run-nixbot-gitlab> gitlab # [    1.827001] fuse: init (API version 7.45)573vm-test-run-nixbot-gitlab> gitlab # [    1.837247] ACPI: \_SB_.LNKC: Enabled at IRQ 10574vm-test-run-nixbot-gitlab> gitlab # [    1.864712] ACPI: \_SB_.LNKA: Enabled at IRQ 10575vm-test-run-nixbot-gitlab> gitlab # [    1.877490] ACPI: \_SB_.LNKB: Enabled at IRQ 11576vm-test-run-nixbot-gitlab> gitlab # [    1.947118] systemd[1]: Successfully made /usr/ read-only.577vm-test-run-nixbot-gitlab> gitlab # [    2.287638] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)578vm-test-run-nixbot-gitlab> gitlab # [    2.310063] systemd[1]: Detected virtualization kvm.579vm-test-run-nixbot-gitlab> gitlab # [    2.314268] systemd[1]: Detected architecture x86-64.580vm-test-run-nixbot-gitlab> gitlab # [    2.318404] systemd[1]: Running in initrd.581vm-test-run-nixbot-gitlab> gitlab # [    2.322761] systemd[1]: Initializing machine ID from random generator.582vm-test-run-nixbot-gitlab> gitlab # [    2.328812] systemd[1]: Hostname set to <gitlab>.583vm-test-run-nixbot-gitlab> gitlab # [    2.412765] systemd[1]: Queued start job for default target Initrd Default Target.584vm-test-run-nixbot-gitlab> gitlab # [    2.556715] systemd[1]: Created slice Slice /system/modprobe.585vm-test-run-nixbot-gitlab> gitlab # [    2.562074] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.586vm-test-run-nixbot-gitlab> gitlab # [    2.568451] systemd[1]: Expecting device /dev/disk/by-label/nix-store...587vm-test-run-nixbot-gitlab> gitlab # [    2.573826] systemd[1]: Expecting device /dev/disk/by-label/nixos...588vm-test-run-nixbot-gitlab> gitlab # [    2.579170] systemd[1]: Reached target Path Units.589vm-test-run-nixbot-gitlab> gitlab # [    2.583420] systemd[1]: Reached target Slice Units.590vm-test-run-nixbot-gitlab> gitlab # [    2.587120] systemd[1]: Reached target Swaps.591vm-test-run-nixbot-gitlab> gitlab # [    2.590162] systemd[1]: Reached target Timer Units.592vm-test-run-nixbot-gitlab> gitlab # [    2.593654] systemd[1]: Listening on D-Bus System Message Bus Socket.593vm-test-run-nixbot-gitlab> gitlab # [    2.598149] systemd[1]: Listening on Journal Socket (/dev/log).594vm-test-run-nixbot-gitlab> gitlab # [    2.602301] systemd[1]: Listening on Journal Sockets.595vm-test-run-nixbot-gitlab> gitlab # [    2.605209] systemd[1]: Listening on udev Control Socket.596vm-test-run-nixbot-gitlab> gitlab # [    2.607073] systemd[1]: Listening on udev Kernel Socket.597vm-test-run-nixbot-gitlab> gitlab # [    2.608791] systemd[1]: Reached target Socket Units.598vm-test-run-nixbot-gitlab> gitlab # [    2.614475] systemd[1]: Starting Create List of Static Device Nodes...599vm-test-run-nixbot-gitlab> gitlab # [    2.620235] systemd[1]: Starting Load Kernel Module 9pnet_virtio...600vm-test-run-nixbot-gitlab> gitlab # [    2.627526] systemd[1]: Starting Load Kernel Module configfs...601vm-test-run-nixbot-gitlab> gitlab # [    2.639837] systemd[1]: Starting Journal Service...602vm-test-run-nixbot-gitlab> gitlab # [    2.648827] systemd[1]: Starting Load Kernel Modules...603vm-test-run-nixbot-gitlab> gitlab # [    2.653149] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki604vm-test-run-nixbot-gitlab> gitlab # [    2.681193] systemd[1]: Starting Coldplug All udev Devices...605vm-test-run-nixbot-gitlab> gitlab # [    2.686562] netfs: FS-Cache loaded606vm-test-run-nixbot-gitlab> gitlab # [    2.696030] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.607vm-test-run-nixbot-gitlab> gitlab # [    2.697362] systemd[1]: Finished Create List of Static Device Nodes.608vm-test-run-nixbot-gitlab> gitlab # [    2.700029] 9pnet: Installing 9P2000 support609vm-test-run-nixbot-gitlab> gitlab # [    2.702271] systemd[1]: modprobe@configfs.service: Deactivated successfully.610vm-test-run-nixbot-gitlab> gitlab # [    2.702695] systemd-journald[152]: Collecting audit messages is disabled.611vm-test-run-nixbot-gitlab> gitlab # [    2.702799] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev612vm-test-run-nixbot-gitlab> gitlab # [    2.715325] systemd[1]: Finished Load Kernel Module configfs.613vm-test-run-nixbot-gitlab> gitlab # [    2.720193] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.614vm-test-run-nixbot-gitlab> gitlab # [    2.724345] systemd[1]: Finished Load Kernel Module 9pnet_virtio.615vm-test-run-nixbot-gitlab> gitlab # [    2.730229] systemd[1]: Finished Load Kernel Modules.616vm-test-run-nixbot-gitlab> gitlab # [    2.732660] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config617vm-test-run-nixbot-gitlab> gitlab # [    2.738297] systemd[1]: Starting Apply Kernel Variables...618vm-test-run-nixbot-gitlab> gitlab # [    2.747051] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...619vm-test-run-nixbot-gitlab> gitlab # [    2.752999] systemd[1]: Finished Apply Kernel Variables.620vm-test-run-nixbot-gitlab> gitlab # [    2.766431] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.621vm-test-run-nixbot-gitlab> gitlab # [    2.512852] s[    2.771074] systemd[1]: Starting Create Static Device Nodes in /dev...622vm-test-run-nixbot-gitlab> gitlab # ystemd-modules-load[154]: Using 4 probe threads[    2.773047] systemd[1]: Started Journal Service.623vm-test-run-nixbot-gitlab> gitlab # 624vm-test-run-nixbot-gitlab> gitlab # [    2.517433] systemd-modules-load[154]: Inserted module 'virtio_balloon'625vm-test-run-nixbot-gitlab> gitlab # [    2.519540] systemd-modules-load[154]: Inserted module 'dm_mod'626vm-test-run-nixbot-gitlab> gitlab # [    2.521416] systemd-modules-load[154]: Inserted module 'virtio_gpu'627vm-test-run-nixbot-gitlab> gitlab # [    2.529255] systemd[1]: Finished Create Static Device Nodes in /dev.628vm-test-run-nixbot-gitlab> gitlab # [    2.531216] systemd[1]: Reached target Preparation for Local File Systems.629vm-test-run-nixbot-gitlab> gitlab # [    2.533161] systemd[1]: Reached target Local File Systems.630vm-test-run-nixbot-gitlab> gitlab # [    2.534706] systemd[1]: Starting Create System Files and Directories...631vm-test-run-nixbot-gitlab> gitlab # [    2.536415] systemd[1]: Starting Rule-based Manager for Device Events and Files...632vm-test-run-nixbot-gitlab> gitlab # [    2.551730] systemd[1]: Finished Create System Files and Directories.633vm-test-run-nixbot-gitlab> gitlab # [    2.557077] systemd-udevd[192]: Using default interface naming scheme 'v260'.634vm-test-run-nixbot-gitlab> gitlab # [    2.571466] systemd[1]: Started Rule-based Manager for Device Events and Files.635vm-test-run-nixbot-gitlab> gitlab # [    2.588618] systemd[1]: Finished Coldplug All udev Devices.636vm-test-run-nixbot-gitlab> gitlab # [    2.590187] systemd[1]: Reached target System Initialization.637vm-test-run-nixbot-gitlab> gitlab # [    2.591743] systemd[1]: Reached target Basic System.638vm-test-run-nixbot-gitlab> gitlab # [    2.991198] virtio_blk virtio4: 4/0/0 default/read/poll queues639vm-test-run-nixbot-gitlab> gitlab # [    2.996816] SCSI subsystem initialized640vm-test-run-nixbot-gitlab> gitlab # [    3.003055] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)641vm-test-run-nixbot-gitlab> gitlab # [    3.004098] uhci_hcd 0000:00:01.2: UHCI Host Controller642vm-test-run-nixbot-gitlab> gitlab # [    3.007145] virtio_blk virtio5: 4/0/0 default/read/poll queues643vm-test-run-nixbot-gitlab> gitlab # [    3.010334] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1644vm-test-run-nixbot-gitlab> gitlab # [    3.012533] uhci_hcd 0000:00:01.2: detected 2 ports645vm-test-run-nixbot-gitlab> gitlab # [    3.014107] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140646vm-test-run-nixbot-gitlab> gitlab # [    3.016270] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18647vm-test-run-nixbot-gitlab> gitlab # [    3.018429] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1648vm-test-run-nixbot-gitlab> gitlab # [    3.020379] usb usb1: Product: UHCI Host Controller649vm-test-run-nixbot-gitlab> gitlab # [    3.021712] usb usb1: Manufacturer: Linux 6.18.34 uhci_hcd650vm-test-run-nixbot-gitlab> gitlab # [    3.023194] usb usb1: SerialNumber: 0000:00:01.2651vm-test-run-nixbot-gitlab> gitlab # [    3.025382] hub 1-0:1.0: USB hub found652vm-test-run-nixbot-gitlab> gitlab # [    3.026517] hub 1-0:1.0: 2 ports detected653vm-test-run-nixbot-gitlab> gitlab # [    3.034746] virtio_blk virtio5: [vdb] 11525992 512-byte logical blocks (5.90 GB/5.50 GiB)654vm-test-run-nixbot-gitlab> gitlab # [    3.038717] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12655vm-test-run-nixbot-gitlab> gitlab # [    3.042215] serio: i8042 KBD port at 0x60,0x64 irq 1656vm-test-run-nixbot-gitlab> gitlab # [    3.043597] serio: i8042 AUX port at 0x60,0x64 irq 12657vm-test-run-nixbot-gitlab> gitlab # [    3.046413] scsi host0: ata_piix658vm-test-run-nixbot-gitlab> gitlab # [    3.047823] scsi host1: ata_piix659vm-test-run-nixbot-gitlab> gitlab # [    3.048839] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0660vm-test-run-nixbot-gitlab> gitlab # [    3.050732] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0661vm-test-run-nixbot-gitlab> gitlab # [    2.823407] (udev-worker)[211]: Network interface NamePolicy= disabled on kernel command line.662vm-test-run-nixbot-gitlab> gitlab # [    2.830566] (udev-worker)[208]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.663vm-test-run-nixbot-gitlab> gitlab # [    2.834726] (udev-worker)[208]: Network interface NamePolicy= disabled on kernel command line.664vm-test-run-nixbot-gitlab> gitlab # [    2.840637] systemd[1]: Starting Virtual Console Setup...665vm-test-run-nixbot-gitlab> gitlab # [    2.853170] systemd[1]: Found device /dev/disk/by-label/nixos.666vm-test-run-nixbot-gitlab> gitlab # [    2.855414] systemd[1]: Reached target Initrd Root Device.667vm-test-run-nixbot-gitlab> gitlab # [    2.857324] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...668vm-test-run-nixbot-gitlab> gitlab # [    2.861235] systemd[1]: Found device /dev/disk/by-label/nix-store.669vm-test-run-nixbot-gitlab> gitlab # [    2.863432] systemd-vconsole-setup[235]: Configuration of first virtual console was skipped, ignoring remaining ones.670vm-test-run-nixbot-gitlab> gitlab # [    2.866221] systemd[1]: Finished Virtual Console Setup.671vm-test-run-nixbot-gitlab> gitlab # [    2.893287] systemd-fsck[243]: nixos: clean, 12/65536 files, 13019/262144 blocks672vm-test-run-nixbot-gitlab> gitlab # [    2.898331] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.673vm-test-run-nixbot-gitlab> gitlab # [    3.206155] ata2: found unknown device (class 0)674vm-test-run-nixbot-gitlab> gitlab # [    3.208711] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100675vm-test-run-nixbot-gitlab> gitlab # [    3.212351] scsi 1:0:0:0: CD-ROM            QEMU     QEMU DVD-ROM     2.5+ PQ: 0 ANSI: 5676vm-test-run-nixbot-gitlab> gitlab # [    3.247171] usb 1-1: new full-speed USB device number 2 using uhci_hcd677vm-test-run-nixbot-gitlab> gitlab # [    3.257645] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0678vm-test-run-nixbot-gitlab> gitlab # [    3.294983] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray679vm-test-run-nixbot-gitlab> gitlab # [    3.339410] cdrom: Uniform CD-ROM driver Revision: 3.20680vm-test-run-nixbot-gitlab> gitlab # [    3.419021] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00681vm-test-run-nixbot-gitlab> gitlab # [    3.423980] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10682vm-test-run-nixbot-gitlab> gitlab # [    3.428390] usb 1-1: Product: QEMU USB Tablet683vm-test-run-nixbot-gitlab> gitlab # [    3.431245] usb 1-1: Manufacturer: QEMU684vm-test-run-nixbot-gitlab> gitlab # [    3.433863] usb 1-1: SerialNumber: 28754-0000:00:01.2-1685vm-test-run-nixbot-gitlab> gitlab # [    3.473363] hid: raw HID events driver (C) Jiri Kosina686vm-test-run-nixbot-gitlab> gitlab # [    3.491045] usbcore: registered new interface driver usbhid687vm-test-run-nixbot-gitlab> gitlab # [    3.494504] usbhid: USB HID core driver688vm-test-run-nixbot-gitlab> gitlab # [    3.507654] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2689vm-test-run-nixbot-gitlab> gitlab # [    3.513585] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0690vm-test-run-nixbot-gitlab> gitlab # [    3.443399] systemd[1]: Mounting /sysroot...691vm-test-run-nixbot-gitlab> gitlab # [    3.873987] EXT4-fs (vda): mounted filesystem 1a2b86b8-d6c6-49de-ae7d-5f550ad9edd2 r/w with ordered data mode. Quota mode: none.692vm-test-run-nixbot-gitlab> gitlab # [    3.621379] systemd[1]: Mounted /sysroot.693vm-test-run-nixbot-gitlab> gitlab # [    3.625248] systemd[1]: Reached target Initrd Root File System.694vm-test-run-nixbot-gitlab> gitlab # [    3.629353] systemd[1]: Mounting /sysroot/nix/.ro-store...695vm-test-run-nixbot-gitlab> gitlab # [    3.633297] systemd[1]: Mounting /sysroot/nix/.rw-store...696vm-test-run-nixbot-gitlab> gitlab # [    3.639120] systemd[1]: Mounting /sysroot/run...697vm-test-run-nixbot-gitlab> gitlab # [    3.644213] systemd[1]: Mounting /sysroot/tmp/shared...698vm-test-run-nixbot-gitlab> gitlab # [    3.909121] erofs (device vdb): mounted with root inode @ nid 128.699vm-test-run-nixbot-gitlab> gitlab # [    3.652104] systemd[1]: Mounting /sysroot/tmp/xchg...700vm-test-run-nixbot-gitlab> gitlab # [    3.662667] systemd[1]: Starting Mountpoints Configured in the Real Root...701vm-test-run-nixbot-gitlab> gitlab # [    3.925207] 9p: Installing v9fs 9p2000 file system support702vm-test-run-nixbot-gitlab> gitlab # [    3.670577] systemd[1]: Mounted /sysroot/nix/.ro-store.703vm-test-run-nixbot-gitlab> gitlab # [    3.672298] systemd[1]: Mounted /sysroot/nix/.rw-store.704vm-test-run-nixbot-gitlab> gitlab # [    3.673907] systemd[1]: Mounted /sysroot/run.705vm-test-run-nixbot-gitlab> gitlab # [    3.675320] systemd[1]: Mounted /sysroot/tmp/xchg.706vm-test-run-nixbot-gitlab> gitlab # [    3.676626] systemd[1]: Mounted /sysroot/tmp/shared.707vm-test-run-nixbot-gitlab> gitlab # [    3.678178] systemd-sysroot-fstab-check[273]: /sysroot should be mounted in the initrd, will request daemon-reload.708vm-test-run-nixbot-gitlab> gitlab # [    3.682139] systemd[1]: Starting rw-sysroot-nix-store.service...709vm-test-run-nixbot-gitlab> gitlab # [    3.683774] systemd[1]: Reload requested from client PID 273 ('systemd-sysroot') (unit initrd-parse-etc.service)...710vm-test-run-nixbot-gitlab> gitlab # [    3.686627] systemd[1]: Reloading...711vm-test-run-nixbot-gitlab> gitlab # [    3.871379] systemd[1]: Reloading finished in 187 ms.712vm-test-run-nixbot-gitlab> gitlab # [    3.927740] systemd-sysroot-fstab-check[273]: Requesting initrd-fs.target/start/replace...713vm-test-run-nixbot-gitlab> gitlab # [    4.042566] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.714vm-test-run-nixbot-gitlab> gitlab # [    4.046693] systemd[1]: Finished rw-sysroot-nix-store.service.715vm-test-run-nixbot-gitlab> gitlab # [    4.059520] systemd-sysroot-fstab-check[273]: Requesting swap.target/start/replace...716vm-test-run-nixbot-gitlab> gitlab # [    4.068258] systemd[1]: initrd-parse-etc.service: Deactivated successfully.717vm-test-run-nixbot-gitlab> gitlab # [    4.072623] systemd[1]: Finished Mountpoints Configured in the Real Root.718vm-test-run-nixbot-gitlab> gitlab # [    4.075941] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.719vm-test-run-nixbot-gitlab> gitlab # [    4.089315] systemd[1]: Starting rw-sysroot-nix-store.service...720vm-test-run-nixbot-gitlab> gitlab # [    4.126290] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.721vm-test-run-nixbot-gitlab> gitlab # [    4.133874] systemd[1]: Finished rw-sysroot-nix-store.service.722vm-test-run-nixbot-gitlab> gitlab # [    4.444843] systemd[1]: Mounting /sysroot/nix/store...723vm-test-run-nixbot-gitlab> gitlab # [    4.499223] systemd[1]: Mounted /sysroot/nix/store.724vm-test-run-nixbot-gitlab> gitlab # [    4.502900] systemd[1]: Reached target Initrd File Systems.725vm-test-run-nixbot-gitlab> gitlab # [    4.508258] systemd[1]: Starting Find NixOS closure...726vm-test-run-nixbot-gitlab> gitlab # [    4.513240] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...727vm-test-run-nixbot-gitlab> gitlab # [    4.547948] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.728vm-test-run-nixbot-gitlab> gitlab # [    4.552837] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.729vm-test-run-nixbot-gitlab> gitlab # [    4.572936] systemd[1]: Finished Find NixOS closure.730vm-test-run-nixbot-gitlab> gitlab # [    4.576331] systemd[1]: Reached target Initrd Default Target.731vm-test-run-nixbot-gitlab> gitlab # [    4.579950] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...732vm-test-run-nixbot-gitlab> gitlab # [    4.619830] systemd[1]: Stopped target Initrd Default Target.733vm-test-run-nixbot-gitlab> gitlab # [    4.624147] systemd[1]: Stopped target Basic System.734vm-test-run-nixbot-gitlab> gitlab # [    4.627572] systemd[1]: Stopped target Initrd Root Device.735vm-test-run-nixbot-gitlab> gitlab # [    4.631285] systemd[1]: Stopped target Path Units.736vm-test-run-nixbot-gitlab> gitlab # [    4.634585] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.737vm-test-run-nixbot-gitlab> gitlab # [    4.639488] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.738vm-test-run-nixbot-gitlab> gitlab # [    4.644342] systemd[1]: Stopped target Slice Units.739vm-test-run-nixbot-gitlab> gitlab # [    4.647660] systemd[1]: Stopped target Socket Units.740vm-test-run-nixbot-gitlab> gitlab # [    4.650918] systemd[1]: Stopped target System Initialization.741vm-test-run-nixbot-gitlab> gitlab # [    4.654488] systemd[1]: Stopped target Swaps.742vm-test-run-nixbot-gitlab> gitlab # [    4.657240] systemd[1]: Stopped target Timer Units.743vm-test-run-nixbot-gitlab> gitlab # [    4.660253] systemd[1]: dbus.socket: Deactivated successfully.744vm-test-run-nixbot-gitlab> gitlab # [    4.663761] systemd[1]: Closed D-Bus System Message Bus Socket.745vm-test-run-nixbot-gitlab> gitlab # [    4.666356] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.746vm-test-run-nixbot-gitlab> gitlab # [    4.669651] systemd[1]: Stopped Find NixOS closure.747vm-test-run-nixbot-gitlab> gitlab # [    4.671904] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio748vm-test-run-nixbot-gitlab> gitlab # [    4.677288] systemd[1]: Starting rw-sysroot-nix-store.service...749vm-test-run-nixbot-gitlab> gitlab # [    4.680537] systemd[1]: systemd-sysctl.service: Deactivated successfully.750vm-test-run-nixbot-gitlab> gitlab # [    4.684347] systemd[1]: Stopped Apply Kernel Variables.751vm-test-run-nixbot-gitlab> gitlab # [    4.686324] systemd[1]: systemd-modules-load.service: Deactivated successfully.752vm-test-run-nixbot-gitlab> gitlab # [    4.688537] systemd[1]: Stopped Load Kernel Modules.753vm-test-run-nixbot-gitlab> gitlab # [    4.690371] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.754vm-test-run-nixbot-gitlab> gitlab # [    4.692652] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.755vm-test-run-nixbot-gitlab> gitlab # [    4.694952] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.756vm-test-run-nixbot-gitlab> gitlab # [    4.697224] systemd[1]: Stopped Create System Files and Directories.757vm-test-run-nixbot-gitlab> gitlab # [    4.699302] systemd[1]: Stopped target Local File Systems.758vm-test-run-nixbot-gitlab> gitlab # [    4.701162] systemd[1]: Stopped target Preparation for Local File Systems.759vm-test-run-nixbot-gitlab> gitlab # [    4.703192] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.760vm-test-run-nixbot-gitlab> gitlab # [    4.706702] systemd[1]: Stopped Coldplug All udev Devices.761vm-test-run-nixbot-gitlab> gitlab # [    4.708352] systemd[1]: Stopping Rule-based Manager for Device Events and Files...762vm-test-run-nixbot-gitlab> gitlab # [    4.710341] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.763vm-test-run-nixbot-gitlab> gitlab # [    4.712146] systemd[1]: Stopped Virtual Console Setup.764vm-test-run-nixbot-gitlab> gitlab # [    4.713497] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.765vm-test-run-nixbot-gitlab> gitlab # [    4.715270] systemd[1]: Finished rw-sysroot-nix-store.service.766vm-test-run-nixbot-gitlab> gitlab # [    4.718441] systemd[1]: systemd-udevd.service: Deactivated successfully.767vm-test-run-nixbot-gitlab> gitlab # [    4.721548] systemd[1]: Stopped Rule-based Manager for Device Events and Files.768vm-test-run-nixbot-gitlab> gitlab # [    4.723268] systemd[1]: systemd-udevd.service: Consumed 1.014s CPU time over 2.182s wall clock time, 26.3M memory peak.769vm-test-run-nixbot-gitlab> gitlab # [    4.728331] systemd[1]: initrd-cleanup.service: Deactivated successfully.770vm-test-run-nixbot-gitlab> gitlab # [    4.731553] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.771vm-test-run-nixbot-gitlab> gitlab # [    4.737561] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.772vm-test-run-nixbot-gitlab> gitlab # [    4.739301] systemd[1]: Closed udev Control Socket.773vm-test-run-nixbot-gitlab> gitlab # [    4.742077] systemd[1]: Starting Cleanup udev Database...774vm-test-run-nixbot-gitlab> gitlab # [    4.743426] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.775vm-test-run-nixbot-gitlab> gitlab # [    4.745261] systemd[1]: Stopped Create Static Device Nodes in /dev.776vm-test-run-nixbot-gitlab> gitlab # [    4.746862] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.777vm-test-run-nixbot-gitlab> gitlab # [    4.748878] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.778vm-test-run-nixbot-gitlab> gitlab # [    4.751769] systemd[1]: kmod-static-nodes.service: Deactivated successfully.779vm-test-run-nixbot-gitlab> gitlab # [    4.753493] systemd[1]: Stopped Create List of Static Device Nodes.780vm-test-run-nixbot-gitlab> gitlab # [    4.763758] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.781vm-test-run-nixbot-gitlab> gitlab # [    4.767232] systemd[1]: Finished Cleanup udev Database.782vm-test-run-nixbot-gitlab> gitlab # [    4.771343] systemd[1]: Reached target Switch Root.783vm-test-run-nixbot-gitlab> gitlab # [    4.774680] systemd[1]: Starting NixOS Activation...784vm-test-run-nixbot-gitlab> gitlab # [    4.870884] initrd-nixos-activation-start[571]: booting system configuration /nix/store/d4fjygpaj45mp5xmmj7mz98m76miz2v3-nixos-system-gitlab-test785vm-test-run-nixbot-gitlab> gitlab # [    4.911297] initrd-nixos-activation-start[571]: running activation script...786vm-test-run-nixbot-gitlab> gitlab # [    5.115679] initrd-nixos-activation-start[594]: setting up /etc...787vm-test-run-nixbot-gitlab> gitlab # [    5.199341] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.788vm-test-run-nixbot-gitlab> gitlab # [    5.206511] systemd[1]: Finished NixOS Activation.789vm-test-run-nixbot-gitlab> gitlab # [    5.220482] systemd[1]: Starting Switch Root...790vm-test-run-nixbot-gitlab> gitlab # [    5.243574] systemd[1]: Switching root.791vm-test-run-nixbot-gitlab> gitlab # [    5.831759] systemd-journald[152]: Received SIGTERM from PID 1 (systemd).792vm-test-run-nixbot-gitlab> gitlab # [    5.960814] NET: Registered PF_VSOCK protocol family793vm-test-run-nixbot-gitlab> gitlab # [    6.342006] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)794vm-test-run-nixbot-gitlab> gitlab # [    6.358749] systemd[1]: Detected virtualization kvm.795vm-test-run-nixbot-gitlab> gitlab # [    6.362008] systemd[1]: Detected architecture x86-64.796vm-test-run-nixbot-gitlab> gitlab # [    6.365261] systemd[1]: Detected first boot.797vm-test-run-nixbot-gitlab> gitlab # [    6.369131] systemd[1]: Initializing machine ID from random generator.798vm-test-run-nixbot-gitlab> gitlab # [    6.506995] systemd[1]: bpf-restrict-fs: LSM BPF program attached799vm-test-run-nixbot-gitlab> gitlab # [    6.569820] systemd[1]: Applying preset policy.800vm-test-run-nixbot-gitlab> gitlab # [    6.721190] systemd[1]: Populated /etc with preset unit settings.801vm-test-run-nixbot-gitlab> gitlab # [    6.828774] systemd[1]: initrd-switch-root.service: Deactivated successfully.802vm-test-run-nixbot-gitlab> gitlab # [    6.831197] systemd[1]: Stopped initrd-switch-root.service.803vm-test-run-nixbot-gitlab> gitlab # [    6.834831] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.804vm-test-run-nixbot-gitlab> gitlab # [    6.838099] systemd[1]: Created slice Slice /system/getty.805vm-test-run-nixbot-gitlab> gitlab # [    6.840276] systemd[1]: Created slice GitLab DevOps Platform Slice.806vm-test-run-nixbot-gitlab> gitlab # [    6.842507] systemd[1]: Created slice User and Session Slice.807vm-test-run-nixbot-gitlab> gitlab # [    6.844131] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.808vm-test-run-nixbot-gitlab> gitlab # [    6.846151] systemd[1]: Started Forward Password Requests to Wall Directory Watch.809vm-test-run-nixbot-gitlab> gitlab # [    6.848025] systemd[1]: Expecting device /dev/hvc0...810vm-test-run-nixbot-gitlab> gitlab # [    6.849366] systemd[1]: Expecting device /dev/ttyS0...811vm-test-run-nixbot-gitlab> gitlab # [    6.850724] systemd[1]: Reached target Local Encrypted Volumes.812vm-test-run-nixbot-gitlab> gitlab # [    6.852269] systemd[1]: Stopped target initrd-fs.target.813vm-test-run-nixbot-gitlab> gitlab # [    6.853651] systemd[1]: Stopped target initrd-root-fs.target.814vm-test-run-nixbot-gitlab> gitlab # [    6.855159] systemd[1]: Stopped target initrd-switch-root.target.815vm-test-run-nixbot-gitlab> gitlab # [    6.856739] systemd[1]: Reached target Virtual Machines and Containers.816vm-test-run-nixbot-gitlab> gitlab # [    6.858456] systemd[1]: Reached target Path Units.817vm-test-run-nixbot-gitlab> gitlab # [    6.859784] systemd[1]: Reached target Remote File Systems.818vm-test-run-nixbot-gitlab> gitlab # [    6.861259] systemd[1]: Reached target Slice Units.819vm-test-run-nixbot-gitlab> gitlab # [    6.862570] systemd[1]: Reached target Swaps.820vm-test-run-nixbot-gitlab> gitlab # [    6.865025] systemd[1]: Listening on Process Core Dump Socket.821vm-test-run-nixbot-gitlab> gitlab # [    6.867164] systemd[1]: Listening on Credential Encryption/Decryption.822vm-test-run-nixbot-gitlab> gitlab # [    6.870744] systemd[1]: Starting Journal Log Access Socket...823vm-test-run-nixbot-gitlab> gitlab # [    6.873075] systemd[1]: Listening on Journal Audit Socket.824vm-test-run-nixbot-gitlab> gitlab # [    6.874947] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.825vm-test-run-nixbot-gitlab> gitlab # [    6.876908] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki826vm-test-run-nixbot-gitlab> gitlab # [    6.879432] systemd[1]: Listening on udev Control Socket.827vm-test-run-nixbot-gitlab> gitlab # [    6.887147] systemd[1]: Mounting Huge Pages File System...828vm-test-run-nixbot-gitlab> gitlab # [    6.890173] systemd[1]: Mounting POSIX Message Queue File System...829vm-test-run-nixbot-gitlab> gitlab # [    6.893328] systemd[1]: Mounting Kernel Debug File System...830vm-test-run-nixbot-gitlab> gitlab # [    6.897980] systemd[1]: Mounting Kernel Trace File System...831vm-test-run-nixbot-gitlab> gitlab # [    6.914819] systemd[1]: Starting Create List of Static Device Nodes...832vm-test-run-nixbot-gitlab> gitlab # [    6.916543] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio833vm-test-run-nixbot-gitlab> gitlab # [    6.921661] systemd[1]: Starting Load Kernel Module configfs...834vm-test-run-nixbot-gitlab> gitlab # [    6.923239] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm835vm-test-run-nixbot-gitlab> gitlab # [    6.925794] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore836vm-test-run-nixbot-gitlab> gitlab # [    6.928782] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse837vm-test-run-nixbot-gitlab> gitlab # [    6.933645] systemd[1]: Mounting FUSE Control File System...838vm-test-run-nixbot-gitlab> gitlab # [    6.948802] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67839vm-test-run-nixbot-gitlab> gitlab # [    6.960158] systemd[1]: Starting Journal Service...840vm-test-run-nixbot-gitlab> gitlab # [    6.967309] systemd[1]: Starting Load Kernel Modules...841vm-test-run-nixbot-gitlab> gitlab # [    6.976331] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...842vm-test-run-nixbot-gitlab> gitlab # [    6.979818] systemd[1]: Starting Remount Root and Kernel File Systems...843vm-test-run-nixbot-gitlab> gitlab # [    6.983494] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki844vm-test-run-nixbot-gitlab> gitlab # [    6.992150] systemd[1]: Starting Coldplug All udev Devices...845vm-test-run-nixbot-gitlab> gitlab # [    7.000200] systemd[1]: Listening on Journal Log Access Socket.846vm-test-run-nixbot-gitlab> gitlab # [    7.001920] systemd[1]: Mounted Huge Pages File System.847vm-test-run-nixbot-gitlab> gitlab # [    7.002121] loop: module loaded848vm-test-run-nixbot-gitlab> gitlab # [    7.002127] systemd[1]: Mounted POSIX Message Queue File System.849vm-test-run-nixbot-gitlab> gitlab # [    7.002348] systemd[1]: Mounted Kernel Debug File System.850vm-test-run-nixbot-gitlab> gitlab # [    7.002955] systemd[1]: Mounted Kernel Trace File System.851vm-test-run-nixbot-gitlab> gitlab # [    7.004950] systemd-journald[790]: Collecting audit messages is enabled.852vm-test-run-nixbot-gitlab> gitlab # [    7.006236] systemd[1]: Finished Create List of Static Device Nodes.853vm-test-run-nixbot-gitlab> gitlab # [    7.012678] systemd[1]: modprobe@configfs.service: Deactivated successfully.854vm-test-run-nixbot-gitlab> gitlab # [    7.017086] EXT4-fs (vda): re-mounted 1a2b86b8-d6c6-49de-ae7d-5f550ad9edd2.855vm-test-run-nixbot-gitlab> gitlab # [    7.020842] systemd[1]: Finished Load Kernel Module configfs.856vm-test-run-nixbot-gitlab> gitlab # [    6.763939] systemd[1]: Queued start job for default target Multi-User System.857vm-test-run-nixbot-gitlab> gitlab # [    7.023726] systemd[1]: Started Journal Service.858vm-test-run-nixbot-gitlab> gitlab # [    6.766693] systemd[1]: Unnecessary job was removed for /dev/vdb.859vm-test-run-nixbot-gitlab> gitlab # [    6.769494] systemd[1]: systemd-journald.service: Deactivated successfully.860vm-test-run-nixbot-gitlab> gitlab # [    6.771657] systemd-modules-load[791]: Using 3 probe threads861vm-test-run-nixbot-gitlab> gitlab # [    6.773400] systemd-modules-load[791]: Inserted module 'tls'862vm-test-run-nixbot-gitlab> gitlab # [    6.775145] systemd-modules-load[791]: Inserted module 'loop'863vm-test-run-nixbot-gitlab> gitlab # [    6.776863] systemd[1]: Mounted FUSE Control File System.864vm-test-run-nixbot-gitlab> gitlab # [    6.778563] systemd[1]: Finished Load Kernel Modules.865vm-test-run-nixbot-gitlab> gitlab # [    6.780176] systemd[1]: Finished Remount Root and Kernel File Systems.866vm-test-run-nixbot-gitlab> gitlab # [    6.783521] systemd-oomd[793]: No swap; memory pressure usage will be degraded867vm-test-run-nixbot-gitlab> gitlab # [    6.786143] systemd[1]: Mounting Kernel Configuration File System...868vm-test-run-nixbot-gitlab> gitlab # [    6.787973] systemd[1]: Starting Firewall...869vm-test-run-nixbot-gitlab> gitlab # [    6.789352] systemd[1]: Starting Flush Journal to Persistent Storage...870vm-test-run-nixbot-gitlab> gitlab # [    6.793131] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore871vm-test-run-nixbot-gitlab> gitlab # [    6.798808] systemd[1]: Starting Load/Save OS Random Seed...872vm-test-run-nixbot-gitlab> gitlab # [    6.800588] systemd[1]: Starting Apply Kernel Variables...873vm-test-run-nixbot-gitlab> gitlab # [    7.061516] systemd-journald[790]: Received client request to flush runtime journal.874vm-test-run-nixbot-gitlab> gitlab # [    6.878122] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...875vm-test-run-nixbot-gitlab> gitlab # [    6.880699] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki876vm-test-run-nixbot-gitlab> gitlab # [    6.883315] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.877vm-test-run-nixbot-gitlab> gitlab # [    6.885320] systemd[1]: Mounted Kernel Configuration File System.878vm-test-run-nixbot-gitlab> gitlab # [    6.887213] systemd[1]: Finished Load/Save OS Random Seed.879vm-test-run-nixbot-gitlab> gitlab # [    6.888798] systemd[1]: Reached target First Boot Complete.880vm-test-run-nixbot-gitlab> gitlab # [    6.890231] systemd[1]: Finished Apply Kernel Variables.881vm-test-run-nixbot-gitlab> gitlab # [    6.891605] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.882vm-test-run-nixbot-gitlab> gitlab # [    6.893338] systemd[1]: Starting Create Static Device Nodes in /dev...883vm-test-run-nixbot-gitlab> gitlab # [    6.894945] systemd[1]: Finished Create Static Device Nodes in /dev.884vm-test-run-nixbot-gitlab> gitlab # [    6.896819] systemd[1]: Reached target Preparation for Local File Systems.885vm-test-run-nixbot-gitlab> gitlab # [    6.898477] systemd[1]: Starting Rule-based Manager for Device Events and Files...886vm-test-run-nixbot-gitlab> gitlab # [    6.901314] systemd[1]: Finished Flush Journal to Persistent Storage.887vm-test-run-nixbot-gitlab> gitlab # [    6.905686] systemd-udevd[830]: Using default interface naming scheme 'v260'.888vm-test-run-nixbot-gitlab> gitlab # [    6.940907] systemd[1]: Finished Coldplug All udev Devices.889vm-test-run-nixbot-gitlab> gitlab # [    6.942830] systemd[1]: Started Rule-based Manager for Device Events and Files.890vm-test-run-nixbot-gitlab> gitlab # [    7.052966] systemd[1]: Finished Firewall.891vm-test-run-nixbot-gitlab> gitlab # [    7.055387] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse892vm-test-run-nixbot-gitlab> gitlab # [    7.092762] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.893vm-test-run-nixbot-gitlab> gitlab # [    7.108275] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped.894vm-test-run-nixbot-gitlab> gitlab # [    7.115618] (udev-worker)[937]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.895vm-test-run-nixbot-gitlab> gitlab # [    7.119492] (udev-worker)[937]: Network interface NamePolicy= disabled on kernel command line.896vm-test-run-nixbot-gitlab> gitlab # [    7.122483] (udev-worker)[921]: Network interface NamePolicy= disabled on kernel command line.897vm-test-run-nixbot-gitlab> gitlab # [    7.141056] systemd[1]: Condition check resulted in Virtio network device being skipped.898vm-test-run-nixbot-gitlab> gitlab # [    7.144392] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore899vm-test-run-nixbot-gitlab> gitlab # [    7.147944] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67900vm-test-run-nixbot-gitlab> gitlab # [    7.152627] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore901vm-test-run-nixbot-gitlab> gitlab # [    7.155924] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki902vm-test-run-nixbot-gitlab> gitlab # [    7.158474] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki903vm-test-run-nixbot-gitlab> gitlab # [    7.424390] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3904vm-test-run-nixbot-gitlab> gitlab # [    7.432985] ACPI: button: Power Button [PWRF]905vm-test-run-nixbot-gitlab> gitlab # [    7.455174] mousedev: PS/2 mouse device common for all mice906vm-test-run-nixbot-gitlab> gitlab # [    7.475193] rtc_cmos 00:05: RTC can wake from S4907vm-test-run-nixbot-gitlab> gitlab # [    7.479369] Floppy drive(s): fd0 is 2.88M AMI BIOS908vm-test-run-nixbot-gitlab> gitlab # [    7.479917] parport_pc 00:03: reported by Plug and Play ACPI909vm-test-run-nixbot-gitlab> gitlab # [    7.486517] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)]910vm-test-run-nixbot-gitlab> gitlab # [    7.488177] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0911vm-test-run-nixbot-gitlab> gitlab # [    7.488267] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD912vm-test-run-nixbot-gitlab> gitlab # [    7.492586] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console913vm-test-run-nixbot-gitlab> gitlab # [    7.493581] FDC 0 is a S82078B914vm-test-run-nixbot-gitlab> gitlab # [    7.495282] rtc_cmos 00:05: registered as rtc0915vm-test-run-nixbot-gitlab> gitlab # [    7.495456] rtc_cmos 00:05: setting system clock to 2026-06-14T14:59:06 UTC (1781449146)916vm-test-run-nixbot-gitlab> gitlab # [    7.498628] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs917vm-test-run-nixbot-gitlab> gitlab # [    7.508052] Console: switching to colour dummy device 80x25918vm-test-run-nixbot-gitlab> gitlab # [    7.509933] systemd-journald[790]: Time jumped backwards, rotating.919vm-test-run-nixbot-gitlab> gitlab # [    7.511394] [drm] Found bochs VGA, ID 0xb0c5.920vm-test-run-nixbot-gitlab> gitlab # [    7.524020] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000.921vm-test-run-nixbot-gitlab> gitlab # [    7.525760] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic922vm-test-run-nixbot-gitlab> gitlab # [    7.527148] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0923vm-test-run-nixbot-gitlab> gitlab # [    7.532463] fbcon: bochs-drmdrmfb (fb0) is primary device924vm-test-run-nixbot-gitlab> gitlab # [    7.542887] Console: switching to colour frame buffer device 160x50925vm-test-run-nixbot-gitlab> gitlab # [    7.545930] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5926vm-test-run-nixbot-gitlab> gitlab # [    7.546806] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device927vm-test-run-nixbot-gitlab> gitlab # [    7.549469] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4928vm-test-run-nixbot-gitlab> gitlab # [    7.668983] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input6929vm-test-run-nixbot-gitlab> gitlab # [    7.691505] ppdev: user-space parallel port driver930vm-test-run-nixbot-gitlab> gitlab # [    7.435328] systemd[1]: Starting Virtual Console Setup...931vm-test-run-nixbot-gitlab> gitlab # [    7.443103] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.932vm-test-run-nixbot-gitlab> gitlab # [    7.445397] systemd[1]: Stopped Virtual Console Setup.933vm-test-run-nixbot-gitlab> gitlab # [    7.704962] kvm_amd: TSC scaling supported934vm-test-run-nixbot-gitlab> gitlab # [    7.705778] kvm_amd: Nested Virtualization enabled935vm-test-run-nixbot-gitlab> gitlab # [    7.447149] s[    7.706975] kvm_amd: Nested Paging enabled936vm-test-run-nixbot-gitlab> gitlab # ystemd[1]: Start[    7.708011] kvm_amd: LBR virtualization supported937vm-test-run-nixbot-gitlab> gitlab # ing Virtual Cons[    7.709305] kvm_amd: Virtual VMLOAD VMSAVE supported938vm-test-run-nixbot-gitlab> gitlab # ole Setup...939vm-test-run-nixbot-gitlab> gitlab # [    7.710500] kvm_amd: Virtual GIF supported940vm-test-run-nixbot-gitlab> gitlab # [    7.714947] kvm_amd: Virtual NMI enabled941vm-test-run-nixbot-gitlab> gitlab # [    7.466303] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.942vm-test-run-nixbot-gitlab> gitlab # [    7.468506] systemd[1]: Stopped Virtual Console Setup.943vm-test-run-nixbot-gitlab> gitlab # [    7.472897] systemd[1]: Starting Virtual Console Setup...944vm-test-run-nixbot-gitlab> gitlab # [    7.770263] EDAC MC: Ver: 3.0.0945vm-test-run-nixbot-gitlab> gitlab # [    7.548853] systemd-vconsole-setup[963]: Configuration of first virtual console was skipped, ignoring remaining ones.946vm-test-run-nixbot-gitlab> gitlab # [    7.551893] systemd[1]: Finished Virtual Console Setup.947vm-test-run-nixbot-gitlab> gitlab # [    7.572601] systemd[1]: Mounting /run/wrappers...948vm-test-run-nixbot-gitlab> gitlab # [    7.598912] systemd[1]: Mounted /run/wrappers.949vm-test-run-nixbot-gitlab> gitlab # [    7.600564] systemd[1]: Reached target Local File Systems.950vm-test-run-nixbot-gitlab> gitlab # [    7.602303] systemd[1]: Listening on Boot Loader Control Service Socket.951vm-test-run-nixbot-gitlab> gitlab # [    7.604329] systemd[1]: Starting register-nix-paths.service...952vm-test-run-nixbot-gitlab> gitlab # [    7.608474] systemd[1]: Starting Create SUID/SGID Wrappers...953vm-test-run-nixbot-gitlab> gitlab # [    7.610485] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.954vm-test-run-nixbot-gitlab> gitlab # [    7.612955] systemd[1]: Starting Save Transient machine-id to Disk...955vm-test-run-nixbot-gitlab> gitlab # [    7.615183] systemd[1]: Starting Create System Files and Directories...956vm-test-run-nixbot-gitlab> gitlab # [    7.641514] systemd[1]: Finished Create System Files and Directories.957vm-test-run-nixbot-gitlab> gitlab # [    7.644174] systemd[1]: Starting Rebuild Journal Catalog...958vm-test-run-nixbot-gitlab> gitlab # [    7.646881] systemd[1]: Starting Record System Boot/Shutdown in UTMP...959vm-test-run-nixbot-gitlab> gitlab # [    7.653232] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.960vm-test-run-nixbot-gitlab> gitlab # [    7.656896] systemd[1]: Finished Save Transient machine-id to Disk.961vm-test-run-nixbot-gitlab> gitlab # [    7.665437] systemd[1]: Finished Record System Boot/Shutdown in UTMP.962vm-test-run-nixbot-gitlab> gitlab # [    7.679911] systemd[1]: Finished Rebuild Journal Catalog.963vm-test-run-nixbot-gitlab> gitlab # [    7.681662] systemd[1]: Starting Update is Completed...964vm-test-run-nixbot-gitlab> gitlab # [    7.695465] systemd[1]: Finished Update is Completed.965vm-test-run-nixbot-gitlab> gitlab # [    7.740215] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.966vm-test-run-nixbot-gitlab> gitlab # [    7.742711] systemd[1]: Finished Create SUID/SGID Wrappers.967vm-test-run-nixbot-gitlab> gitlab # [    7.744131] systemd[1]: suid-sgid-wrappers.service: Consumed 106ms CPU time over 136ms wall clock time, 4.8M memory peak, 1.1M read from disk.968vm-test-run-nixbot-gitlab> gitlab # [    8.006842] systemd[1]: Finished register-nix-paths.service.969vm-test-run-nixbot-gitlab> gitlab # [    8.010622] systemd[1]: Reached target System Initialization.970vm-test-run-nixbot-gitlab> gitlab # [    8.014405] systemd[1]: Started Discard unused filesystem blocks once a week.971vm-test-run-nixbot-gitlab> gitlab # [    8.018672] systemd[1]: Started logrotate.timer.972vm-test-run-nixbot-gitlab> gitlab # [    8.021752] systemd[1]: Started Daily Cleanup of Temporary Directories.973vm-test-run-nixbot-gitlab> gitlab # [    8.025870] systemd[1]: Reached target Timer Units.974vm-test-run-nixbot-gitlab> gitlab # [    8.029231] systemd[1]: Listening on D-Bus System Message Bus Socket.975vm-test-run-nixbot-gitlab> gitlab # [    8.033303] systemd[1]: Listening on Nix Daemon Socket.976vm-test-run-nixbot-gitlab> gitlab # [    8.036641] systemd[1]: Starting nixbot CI service socket...977vm-test-run-nixbot-gitlab> gitlab # [    8.040457] systemd[1]: Listening on Hostname Service Socket.978vm-test-run-nixbot-gitlab> gitlab # [    8.044670] systemd[1]: Starting D-Bus System Message Bus...979vm-test-run-nixbot-gitlab> gitlab # [    8.048471] systemd[1]: Listening on nixbot CI service socket.980vm-test-run-nixbot-gitlab> gitlab # [    8.052145] systemd[1]: Reached target Socket Units.981vm-test-run-nixbot-gitlab> gitlab # [    8.091189] dbus-broker-launch[1054]: Looking up NSS user entry for 'systemd-timesync'...982vm-test-run-nixbot-gitlab> gitlab # [    8.096981] dbus-broker-launch[1054]: NSS returned no entry for 'systemd-timesync'983vm-test-run-nixbot-gitlab> gitlab # [    8.101647] dbus-broker-launch[1054]: Invalid user-name in /nix/store/89zj9nmrajvcy9c3c08wv0ir9ww7lgmq-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"984vm-test-run-nixbot-gitlab> gitlab # [    8.110796] systemd[1]: Started D-Bus System Message Bus.985vm-test-run-nixbot-gitlab> gitlab # [    8.130655] dbus-broker-launch[1054]: Ready986vm-test-run-nixbot-gitlab> gitlab # [    8.138217] systemd[1]: Reached target Basic System.987vm-test-run-nixbot-gitlab> gitlab # [    8.141918] systemd[1]: Started backdoor.service.988vm-test-run-nixbot-gitlab> gitlab # [    8.145624] systemd[1]: Starting gitlab-config.service...989vm-test-run-nixbot-gitlab> gitlab # [    8.149529] systemd[1]: Starting Import lastlog data into lastlog2 database...990vm-test-run-nixbot-gitlab> gitlab # [    8.167475] systemd[1]: Starting Logrotate configuration check...991vm-test-run-nixbot-gitlab> gitlab # [    8.174353] systemd[1]: Starting Name Service Cache Daemon (nsncd)...992vm-test-run-nixbot-gitlab> gitlab # [    8.180151] systemd[1]: Starting Post-Boot Actions...993vm-test-run-nixbot-gitlab> gitlab # connecting to host...994vm-test-run-nixbot-gitlab> gitlab # [    8.184652] systemd[1]: Started Reset console on configuration changes.995vm-test-run-nixbot-gitlab> gitlab # [    8.190641] systemd[1]: Starting resolvconf update...996vm-test-run-nixbot-gitlab> gitlab: Guest shell says: b'Spawning backdoor root shell...\n'997vm-test-run-nixbot-gitlab> gitlab: connected to guest root shell998vm-test-run-nixbot-gitlab> gitlab: (connecting took 9.03 seconds)999vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for the VM to finish booting, in 9.03 seconds)1000vm-test-run-nixbot-gitlab> gitlab # [    8.194171] logrotate[1059]: warning: logrotate in debug mode does nothing except printing debug messages!  Consider using verbose mode (-v) instead if this is not what you want.1001vm-test-run-nixbot-gitlab> gitlab # [    8.201179] logrotate[1059]: reading config file /nix/store/sn4bd2nqi3yg4r51djm9i2rmfgbwvmhb-logrotate.conf1002vm-test-run-nixbot-gitlab> gitlab # [    8.203581] logrotate[1059]: note: 'monthly' overrides previously specified 'weekly'1003vm-test-run-nixbot-gitlab> gitlab # [    8.205453] logrotate[1059]: Reading state from file: /var/lib/logrotate.status1004vm-test-run-nixbot-gitlab> gitlab # [    8.207296] logrotate[1059]: state file /var/lib/logrotate.status does not exist1005vm-test-run-nixbot-gitlab> gitlab # [    8.209274] logrotate[1059]: Allocating hash table for state file, size 64 entries1006vm-test-run-nixbot-gitlab> gitlab # [    8.211511] logrotate[1059]: Handling 4 logs1007vm-test-run-nixbot-gitlab> gitlab # [    8.212671] logrotate[1059]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed1008vm-test-run-nixbot-gitlab> gitlab # [    8.216435] logrotate[1059]: considering log /var/log/btmp1009vm-test-run-nixbot-gitlab> gitlab # [    8.217859] logrotate[1059]: Creating new state1010vm-test-run-nixbot-gitlab> gitlab # [    8.219188] logrotate[1059]:   Now: 2026-06-14 14:591011vm-test-run-nixbot-gitlab> gitlab # [    8.220668] logrotate[1059]:   Last rotated at 2026-06-14 14:001012vm-test-run-nixbot-gitlab> gitlab # [    8.222571] logrotate[1059]:   log does not need rotating (log has already been rotated)1013vm-test-run-nixbot-gitlab> gitlab # [    8.224649] logrotate[1059]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed1014vm-test-run-nixbot-gitlab> gitlab # [    8.229242] logrotate[1059]: considering log /var/log/wtmp1015vm-test-run-nixbot-gitlab> gitlab # [    8.231259] logrotate[1059]: Creating new state1016vm-test-run-nixbot-gitlab> gitlab # [    8.232539] logrotate[1059]:   Now: 2026-06-14 14:591017vm-test-run-nixbot-gitlab> gitlab # [    8.233900] logrotate[1059]:   Last rotated at 2026-06-14 14:001018vm-test-run-nixbot-gitlab> gitlab # [    8.235442] logrotate[1059]:   log does not need rotating (log has already been rotated)1019vm-test-run-nixbot-gitlab> gitlab # [    8.237411] logrotate[1059]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed1020vm-test-run-nixbot-gitlab> gitlab # [    8.240734] logrotate[1059]: switching euid from 0 to 165 and egid from 0 to 165 (pid 1059)1021vm-test-run-nixbot-gitlab> gitlab # [    8.242772] logrotate[1059]: considering log /var/gitlab/state/log/*.log1022vm-test-run-nixbot-gitlab> gitlab # [    8.244387] logrotate[1059]:   log /var/gitlab/state/log/*.log does not exist -- skipping1023vm-test-run-nixbot-gitlab> gitlab # [    8.246428] logrotate[1059]: Creating new state1024vm-test-run-nixbot-gitlab> gitlab # [    8.247658] logrotate[1059]: switching euid from 165 to 0 and egid from 165 to 0 (pid 1059)1025vm-test-run-nixbot-gitlab> gitlab # [    8.249705] logrotate[1059]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed1026vm-test-run-nixbot-gitlab> gitlab # [    8.252756] logrotate[1059]: switching euid from 0 to 60 and egid from 0 to 60 (pid 1059)1027vm-test-run-nixbot-gitlab> gitlab # [    8.254747] logrotate[1059]: considering log /var/log/nginx/*.log1028vm-test-run-nixbot-gitlab> gitlab # [    8.256625] logrotate[1059]:   log /var/log/nginx/*.log does not exist -- skipping1029vm-test-run-nixbot-gitlab> gitlab # [    8.259308] logrotate[1059]: Creating new state1030vm-test-run-nixbot-gitlab> gitlab # [    8.260105] logrotate[1059]: switching euid from 60 to 0 and egid from 60 to 0 (pid 1059)1031vm-test-run-nixbot-gitlab> gitlab # [    8.261441] nsncd[1063]: Jun 14 14:59:07.469 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"1032vm-test-run-nixbot-gitlab> gitlab # [    8.263070] systemd[1]: Finished Logrotate configuration check.1033vm-test-run-nixbot-gitlab> gitlab # [    8.266989] systemd[1]: Finished Post-Boot Actions.1034vm-test-run-nixbot-gitlab> gitlab # [    8.268312] systemd[1]: Started Name Service Cache Daemon (nsncd).1035vm-test-run-nixbot-gitlab> gitlab # [    8.269632] systemd[1]: Finished Import lastlog data into lastlog2 database.1036vm-test-run-nixbot-gitlab> gitlab # [    8.271530] systemd[1]: Reached target Host and Network Name Lookups.1037vm-test-run-nixbot-gitlab> gitlab # [    8.272616] systemd[1]: Reached target User and Group Name Lookups.1038vm-test-run-nixbot-gitlab> gitlab # [    8.274550] systemd[1]: Starting User Login Management...1039vm-test-run-nixbot-gitlab> gitlab # [    8.282914] systemd-logind[1097]: New seat seat0.1040vm-test-run-nixbot-gitlab> gitlab # [    8.286126] systemd-logind[1097]: Watching system buttons on /dev/input/event2 (Power Button)1041vm-test-run-nixbot-gitlab> gitlab # [    8.287104] systemd-logind[1097]: Watching system buttons on /dev/input/event5 (QEMU Virtio Keyboard)1042vm-test-run-nixbot-gitlab> gitlab # [    8.287869] systemd-logind[1097]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard)1043vm-test-run-nixbot-gitlab> gitlab # [    8.288740] systemd[1]: Started User Login Management.1044vm-test-run-nixbot-gitlab> gitlab # [    8.291072] systemd[1]: Stopped target Host and Network Name Lookups.1045vm-test-run-nixbot-gitlab> gitlab # [    8.292067] systemd[1]: Stopping Host and Network Name Lookups...1046vm-test-run-nixbot-gitlab> gitlab # [    8.292935] systemd[1]: Stopped target User and Group Name Lookups.1047vm-test-run-nixbot-gitlab> gitlab # [    8.294167] systemd[1]: Stopping User and Group Name Lookups...1048vm-test-run-nixbot-gitlab> gitlab # [    8.297332] systemd[1]: Starting linger-users.service...1049vm-test-run-nixbot-gitlab> gitlab # [    8.298799] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...1050vm-test-run-nixbot-gitlab> gitlab # [    8.305072] systemd[1]: nscd.service: Deactivated successfully.1051vm-test-run-nixbot-gitlab> gitlab # [    8.305878] systemd[1]: Stopped Name Service Cache Daemon (nsncd).1052vm-test-run-nixbot-gitlab> gitlab # [    8.309460] systemd[1]: nscd.service: Consumed 17ms CPU time over 131ms wall clock time, 7.8M memory peak, 5.6M read from disk.1053vm-test-run-nixbot-gitlab> gitlab # [    8.313983] systemd[1]: Starting Name Service Cache Daemon (nsncd)...1054vm-test-run-nixbot-gitlab> gitlab # [    8.319436] systemd[1]: linger-users.service: Deactivated successfully.1055vm-test-run-nixbot-gitlab> gitlab # [    8.319979] systemd[1]: Finished linger-users.service.1056vm-test-run-nixbot-gitlab> gitlab # [    8.334576] nsncd[1147]: Jun 14 14:59:07.596 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"1057vm-test-run-nixbot-gitlab> gitlab # [    8.338386] systemd[1]: Started Name Service Cache Daemon (nsncd).1058vm-test-run-nixbot-gitlab> gitlab # [    8.338916] systemd[1]: Reached target Host and Network Name Lookups.1059vm-test-run-nixbot-gitlab> gitlab # [    8.340123] systemd[1]: Reached target User and Group Name Lookups.1060vm-test-run-nixbot-gitlab> gitlab # [    8.343656] systemd[1]: Finished resolvconf update.1061vm-test-run-nixbot-gitlab> gitlab # [    8.345192] systemd[1]: Reached target Preparation for Network.1062vm-test-run-nixbot-gitlab> gitlab # [    8.347100] systemd[1]: Starting DHCP Client...1063vm-test-run-nixbot-gitlab> gitlab # [    8.350294] systemd[1]: Starting Address configuration of eth1...1064vm-test-run-nixbot-gitlab> gitlab # [    8.353525] systemd[1]: Starting Extra networking commands....1065vm-test-run-nixbot-gitlab> gitlab # [    8.376361] network-addresses-eth1-start[1172]: adding address 192.168.1.1/24... done1066vm-test-run-nixbot-gitlab> gitlab # [    8.379322] network-addresses-eth1-start[1172]: adding address 2001:db8:1::1/64... done1067vm-test-run-nixbot-gitlab> gitlab # [    8.384816] systemd[1]: Finished Address configuration of eth1.1068vm-test-run-nixbot-gitlab> gitlab # [    8.415372] systemd[1]: Finished Extra networking commands..1069vm-test-run-nixbot-gitlab> gitlab # [    8.418282] systemd[1]: Reached target Network.1070vm-test-run-nixbot-gitlab> gitlab # [    8.420122] systemd[1]: Starting gitlab-workhorse.service...1071vm-test-run-nixbot-gitlab> gitlab # [    8.423305] systemd[1]: Starting Nginx Web Server...1072vm-test-run-nixbot-gitlab> gitlab # [    8.424415] dhcpcd[1186]: dhcpcd-10.3.2 starting1073vm-test-run-nixbot-gitlab> gitlab # [    8.428092] systemd[1]: Starting PostgreSQL Server...1074vm-test-run-nixbot-gitlab> gitlab # [    8.429198] dhcpcd[1232]: dev: loaded udev1075vm-test-run-nixbot-gitlab> gitlab # [    8.430279] systemd[1]: Starting Redis Server - redis-gitlab...1076vm-test-run-nixbot-gitlab> gitlab # [    8.693658] 8021q: 802.1Q VLAN Support v1.81077vm-test-run-nixbot-gitlab> gitlab # [    8.436193] s[    8.694836] 8021q: adding VLAN 0 to HW filter on device eth11078vm-test-run-nixbot-gitlab> gitlab # ystemd[1]: Starting Permit User Sessions...1079vm-test-run-nixbot-gitlab> gitlab # [    8.452105] systemd[1]: Finished Permit User Sessions.1080vm-test-run-nixbot-gitlab> gitlab # [    8.464576] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.1081vm-test-run-nixbot-gitlab> gitlab # [    8.468462] systemd[1]: Started Getty on tty1.1082vm-test-run-nixbot-gitlab> gitlab # [    8.469572] systemd[1]: Reached target Login Prompts.1083vm-test-run-nixbot-gitlab> gitlab # [    8.778139] cfg80211: Loading compiled-in X.509 certificates for regulatory database1084vm-test-run-nixbot-gitlab> gitlab # [    8.784331] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'1085vm-test-run-nixbot-gitlab> gitlab # [    8.786245] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'1086vm-test-run-nixbot-gitlab> gitlab # [    8.788556] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -21087vm-test-run-nixbot-gitlab> gitlab # [    8.790562] cfg80211: failed to load regulatory.db1088vm-test-run-nixbot-gitlab> gitlab # [    8.827362] 8021q: adding VLAN 0 to HW filter on device eth01089vm-test-run-nixbot-gitlab> gitlab # [    8.572990] dhcpcd[1232]: eth0: waiting for carrier1090vm-test-run-nixbot-gitlab> gitlab # [    8.574834] dhcpcd[1232]: eth0: carrier acquired1091vm-test-run-nixbot-gitlab> gitlab # [    8.580523] dhcpcd[1232]: DUID 00:01:00:01:31:c1:7e:3b:52:54:00:12:34:561092vm-test-run-nixbot-gitlab> gitlab # [    8.581177] dhcpcd[1232]: eth0: IAID 00:12:34:561093vm-test-run-nixbot-gitlab> gitlab # [    8.582306] dhcpcd[1232]: eth0: adding address fe80::5054:ff:fe12:34561094vm-test-run-nixbot-gitlab> gitlab # [    8.600900] nginx-pre-start[1258]: nginx: the configuration file /nix/store/nlpb2pnbsk0921apkv5yml2qxbm30d7w-nginx.conf syntax is ok1095vm-test-run-nixbot-gitlab> gitlab # [    8.604217] nginx-pre-start[1258]: nginx: configuration file /nix/store/nlpb2pnbsk0921apkv5yml2qxbm30d7w-nginx.conf test is successful1096vm-test-run-nixbot-gitlab> gitlab # [    8.607618] redis-server[1253]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.1097vm-test-run-nixbot-gitlab> gitlab # [    8.612115] systemd[1]: Started Redis Server - redis-gitlab.1098vm-test-run-nixbot-gitlab> gitlab # [    8.614199] postgresql-pre-start[1262]: The files belonging to this database system will be owned by user "postgres".1099vm-test-run-nixbot-gitlab> gitlab # [    8.617130] postgresql-pre-start[1262]: This user must also own the server process.1100vm-test-run-nixbot-gitlab> gitlab # [    8.619292] redis-server[1253]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo1101vm-test-run-nixbot-gitlab> gitlab # [    8.621563] systemd[1]: Started Nginx Web Server.1102vm-test-run-nixbot-gitlab> gitlab # [    8.622938] postgresql-pre-start[1262]: The database cluster will be initialized with locale "en_US.UTF-8".1103vm-test-run-nixbot-gitlab> gitlab # [    8.625337] postgresql-pre-start[1262]: The default database encoding has accordingly been set to "UTF8".1104vm-test-run-nixbot-gitlab> gitlab # [    8.627867] postgresql-pre-start[1262]: The default text search configuration will be set to "english".1105vm-test-run-nixbot-gitlab> gitlab # [    8.630409] postgresql-pre-start[1262]: Data page checksums are disabled.1106vm-test-run-nixbot-gitlab> gitlab # [    8.632336] postgresql-pre-start[1262]: fixing permissions on existing directory /var/lib/postgresql/17 ... ok1107vm-test-run-nixbot-gitlab> gitlab # [    8.635080] postgresql-pre-start[1262]: creating subdirectories ... ok1108vm-test-run-nixbot-gitlab> gitlab # [    8.637266] postgresql-pre-start[1262]: selecting dynamic shared memory implementation ... posix1109vm-test-run-nixbot-gitlab> gitlab # [    8.639531] redis-server[1253]: Redis version=8.8.0, bits=64, commit=00000000, modified=0, pid=1253, just started1110vm-test-run-nixbot-gitlab> gitlab # [    8.642399] redis-server[1253]: Configuration loaded1111vm-test-run-nixbot-gitlab> gitlab # [    8.644122] redis[1253]: monotonic clock: POSIX clock_gettime1112vm-test-run-nixbot-gitlab> gitlab # [    8.646313] redis[1253]: Running mode=standalone, port=0.1113vm-test-run-nixbot-gitlab> gitlab # [    8.648256] redis[1253]: Server initialized1114vm-test-run-nixbot-gitlab> gitlab # [    8.649920] redis[1253]: Ready to accept connections unix1115vm-test-run-nixbot-gitlab> gitlab # [    8.651932] redis[1253]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.1116vm-test-run-nixbot-gitlab> gitlab # [    8.655398] postgresql-pre-start[1262]: selecting default "max_connections" ... 1001117vm-test-run-nixbot-gitlab> gitlab # [    8.693296] postgresql-pre-start[1262]: selecting default "shared_buffers" ... 128MB1118vm-test-run-nixbot-gitlab> gitlab # [    8.881649] systemd[1]: Started gitlab-workhorse.service.1119vm-test-run-nixbot-gitlab> gitlab # [    9.052508] postgresql-pre-start[1262]: selecting default time zone ... UTC1120vm-test-run-nixbot-gitlab> gitlab # [    9.054750] postgresql-pre-start[1262]: creating configuration files ... ok1121vm-test-run-nixbot-gitlab> gitlab # [    9.117225] gitlab-workhorse[1290]: time="2026-06-14T14:59:08Z" level=info msg=Starting build_time=19700101.000000 version=18.11.41122vm-test-run-nixbot-gitlab> gitlab # [    9.118142] gitlab-workhorse[1290]: time="2026-06-14T14:59:08Z" level=info msg="Using redis/go-redis"1123vm-test-run-nixbot-gitlab> gitlab # [    9.119556] gitlab-workhorse[1290]: time="2026-06-14T14:59:08Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix1124vm-test-run-nixbot-gitlab> gitlab # [    9.204245] postgresql-pre-start[1262]: running bootstrap script ... ok1125vm-test-run-nixbot-gitlab> gitlab # [    9.333562] dhcpcd[1232]: eth0: soliciting a DHCP lease1126vm-test-run-nixbot-gitlab> gitlab # [    9.596363] NET: Registered PF_PACKET protocol family1127vm-test-run-nixbot-gitlab> gitlab # [    9.345914] dhcpcd[1232]: eth0: offered 10.0.2.15 from 10.0.2.21128vm-test-run-nixbot-gitlab> gitlab # [    9.352526] dhcpcd[1232]: eth0: probing address 10.0.2.15/241129vm-test-run-nixbot-gitlab> gitlab # [    9.675325] postgresql-pre-start[1262]: performing post-bootstrap initialization ... ok1130vm-test-run-nixbot-gitlab> gitlab # [    9.959254] postgresql-pre-start[1262]: syncing data to disk ... ok1131vm-test-run-nixbot-gitlab> gitlab # [    9.960464] postgresql-pre-start[1262]: initdb: warning: enabling "trust" authentication for local connections1132vm-test-run-nixbot-gitlab> gitlab # [    9.961834] postgresql-pre-start[1262]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.1133vm-test-run-nixbot-gitlab> gitlab # [    9.963491] postgresql-pre-start[1262]: Success. You can now start the database server using:1134vm-test-run-nixbot-gitlab> gitlab # [    9.964977] postgresql-pre-start[1262]:     pg_ctl -D /var/lib/postgresql/17 -l logfile start1135vm-test-run-nixbot-gitlab> gitlab # [   10.066748] postgres[1315]: [1315] LOG:  starting PostgreSQL 17.10 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit1136vm-test-run-nixbot-gitlab> gitlab # [   10.068529] postgres[1315]: [1315] LOG:  listening on IPv6 address "::1", port 54321137vm-test-run-nixbot-gitlab> gitlab # [   10.070267] postgres[1315]: [1315] LOG:  listening on IPv4 address "127.0.0.1", port 54321138vm-test-run-nixbot-gitlab> gitlab # [   10.072435] postgres[1315]: [1315] LOG:  listening on Unix socket "/run/postgresql/.s.PGSQL.5432"1139vm-test-run-nixbot-gitlab> gitlab # [   10.080369] postgres[1322]: [1322] LOG:  database system was shut down at 2026-06-14 14:59:08 GMT1140vm-test-run-nixbot-gitlab> gitlab # [   10.089080] postgres[1315]: [1315] LOG:  database system is ready to accept connections1141vm-test-run-nixbot-gitlab> gitlab # [   10.094180] systemd[1]: Started PostgreSQL Server.1142vm-test-run-nixbot-gitlab> gitlab # [   10.099140] systemd[1]: Starting PostgreSQL Setup Scripts...1143vm-test-run-nixbot-gitlab> gitlab # [   10.277995] postgresql-setup-start[1333]: CREATE DATABASE1144vm-test-run-nixbot-gitlab> gitlab # [   10.313168] postgresql-setup-start[1338]: CREATE ROLE1145vm-test-run-nixbot-gitlab> gitlab # [   10.334803] postgresql-setup-start[1344]: ALTER DATABASE1146vm-test-run-nixbot-gitlab> gitlab # [   10.360447] postgresql-setup-start[1354]: CREATE ROLE1147vm-test-run-nixbot-gitlab> gitlab # [   10.362631] systemd[1]: Finished PostgreSQL Setup Scripts.1148vm-test-run-nixbot-gitlab> gitlab # [   10.364300] systemd[1]: Reached target PostgreSQL.1149vm-test-run-nixbot-gitlab> gitlab # [   10.366725] systemd[1]: Starting gitlab-postgresql.service...1150vm-test-run-nixbot-gitlab> gitlab # [   10.473198] gitlab-postgresql-start[1362]: CREATE DATABASE1151vm-test-run-nixbot-gitlab> gitlab # [   10.519552] gitlab-postgresql-start[1371]: CREATE EXTENSION1152vm-test-run-nixbot-gitlab> gitlab # [   10.525228] 119c5p75zqwmpdrsjsd5cszxwrq2hgfq-gitlab-config[1367]: mkdir -p /var/gitlab/state/home/.ssh: OK1153vm-test-run-nixbot-gitlab> gitlab # [   10.527522] 119c5p75zqwmpdrsjsd5cszxwrq2hgfq-gitlab-config[1367]: chmod 700 /var/gitlab/state/home/.ssh: OK1154vm-test-run-nixbot-gitlab> gitlab # [   10.564992] systemd[1]: Finished gitlab-config.service.1155vm-test-run-nixbot-gitlab> gitlab # [   10.568235] systemd[1]: Started gitaly.service.1156vm-test-run-nixbot-gitlab> gitlab # [   10.570112] systemd[1]: Starting gitlab-db-config.service...1157vm-test-run-nixbot-gitlab> gitlab # [   10.578937] gitlab-postgresql-start[1374]: CREATE EXTENSION1158vm-test-run-nixbot-gitlab> gitlab # [   10.583277] systemd[1]: Finished gitlab-postgresql.service.1159vm-test-run-nixbot-gitlab> gitlab # [   11.148509] gitaly[1387]: time="2026-06-14T14:59:10.409Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/2wcq0nlg508fs5wq7jnpg1lw2pib950g-gitaly.toml' and fix the errors that are printed." pid=13871160vm-test-run-nixbot-gitlab> gitlab # [   11.152691] gitaly[1387]: time="2026-06-14T14:59:10.410Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=13871161vm-test-run-nixbot-gitlab> gitlab # [   11.154549] gitaly[1387]: time="2026-06-14T14:59:10.410Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=13871162vm-test-run-nixbot-gitlab> gitlab # [   11.155876] gitaly[1387]: time="2026-06-14T14:59:10.410Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=13871163vm-test-run-nixbot-gitlab> gitlab # [   11.157442] gitaly[1387]: time="2026-06-14T14:59:10.410Z" level=info msg="Starting Gitaly" pid=1387 version=18.11.41164vm-test-run-nixbot-gitlab> gitlab # [   11.270622] dhcpcd[1232]: eth0: soliciting an IPv6 router1165vm-test-run-nixbot-gitlab> gitlab # [   11.271307] dhcpcd[1232]: eth0: Router Advertisement from fe80::21166vm-test-run-nixbot-gitlab> gitlab # [   11.272371] dhcpcd[1232]: eth0: adding address fec0::5054:ff:fe12:3456/641167vm-test-run-nixbot-gitlab> gitlab # [   11.273991] dhcpcd[1232]: eth0: adding route to fec0::/641168vm-test-run-nixbot-gitlab> gitlab # [   11.275149] dhcpcd[1232]: eth0: adding default route via fe80::21169vm-test-run-nixbot-gitlab> gitlab # [   11.356180] gitaly[1387]: time="2026-06-14T14:59:10.618Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=207 pid=13871170vm-test-run-nixbot-gitlab> gitlab # [   11.356926] gitaly[1387]: time="2026-06-14T14:59:10.618Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=13871171vm-test-run-nixbot-gitlab> gitlab # [   11.358233] gitaly[1387]: time="2026-06-14T14:59:10.618Z" level=info msg="finished initializing command factory" duration_ms=0 pid=13871172vm-test-run-nixbot-gitlab> gitlab # [   11.359106] gitaly[1387]: time="2026-06-14T14:59:10.618Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-1387/git-exec-1412986810.d/git pid=13871173vm-test-run-nixbot-gitlab> gitlab # [   11.362504] gitaly[1387]: time="2026-06-14T14:59:10.624Z" level=info msg="finished detecting git version" duration_ms=5 pid=13871174vm-test-run-nixbot-gitlab> gitlab # [   11.363316] gitaly[1387]: time="2026-06-14T14:59:10.624Z" level=info msg="using Git version" pid=1387 version=2.53.g1175vm-test-run-nixbot-gitlab> gitlab # [   11.364247] gitaly[1387]: time="2026-06-14T14:59:10.625Z" level=info msg="clearing disk cache object folder" pid=1387 storage=default1176vm-test-run-nixbot-gitlab> gitlab # [   11.364754] gitaly[1387]: time="2026-06-14T14:59:10.625Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache4171465330 pid=1387 storage=default1177vm-test-run-nixbot-gitlab> gitlab # [   11.365904] gitaly[1387]: time="2026-06-14T14:59:10.625Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache4171465330 pid=1387 storage=default1178vm-test-run-nixbot-gitlab> gitlab # [   11.367325] gitaly[1387]: time="2026-06-14T14:59:10.625Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=13871179vm-test-run-nixbot-gitlab> gitlab # [   11.368320] gitaly[1387]: time="2026-06-14T14:59:10.625Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=13871180vm-test-run-nixbot-gitlab> gitlab # [   11.369431] gitaly[1387]: time="2026-06-14T14:59:10.626Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache4171465330 clear_duration_ms=0 pid=1387 storage=default1181vm-test-run-nixbot-gitlab> gitlab # [   11.438377] gitaly[1387]: time="2026-06-14T14:59:10.700Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=13871182vm-test-run-nixbot-gitlab> gitlab # [   11.438884] gitaly[1387]: time="2026-06-14T14:59:10.700Z" level=info msg="counting repositories" pid=13871183vm-test-run-nixbot-gitlab> gitlab # [   11.439516] gitaly[1387]: time="2026-06-14T14:59:10.700Z" level=info msg="starting to count repositories" pid=1387 storage_path=/var/gitlab/state/repositories1184vm-test-run-nixbot-gitlab> gitlab # [   11.440899] gitaly[1387]: time="2026-06-14T14:59:10.700Z" level=info msg="finished tempdir cleaner walk" pid=1387 storage=default time_ms=01185vm-test-run-nixbot-gitlab> gitlab # [   11.441985] gitaly[1387]: time="2026-06-14T14:59:10.700Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=1387 storage_path=/var/gitlab/state/repositories1186vm-test-run-nixbot-gitlab> gitlab # [   11.443149] gitaly[1387]: time="2026-06-14T14:59:10.700Z" level=info msg="completed counting all repositories" counting_duration=0 pid=13871187vm-test-run-nixbot-gitlab> gitlab # [   11.443995] gitaly[1387]: time="2026-06-14T14:59:10.700Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=1387 schema=unix1188vm-test-run-nixbot-gitlab> gitlab # [   11.444861] gitaly[1387]: time="2026-06-14T14:59:10.701Z" level=info msg="listening at address" address=/run/gitaly/gitaly-1387/sock.d/intern pid=1387 schema=unix1189vm-test-run-nixbot-gitlab> gitlab # [   11.465715] gitaly[1387]: time="2026-06-14T14:59:10.727Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=13871190vm-test-run-nixbot-gitlab> gitlab # [   11.466255] gitaly[1387]: time="2026-06-14T14:59:10.728Z" level=info msg="Started Gitaly" duration_ms=317 pid=13871191vm-test-run-nixbot-gitlab> gitlab # [   11.467071] gitaly[1387]: time="2026-06-14T14:59:10.728Z" level=info msg="maintenance: daily scheduled" pid=1387 scheduled="2026-06-15 12:00:00 +0000 UTC"1192vm-test-run-nixbot-gitlab> gitlab # [   11.857583] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1391]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1193vm-test-run-nixbot-gitlab> gitlab # [   11.859538] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1391]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1194vm-test-run-nixbot-gitlab> gitlab # [   11.880389] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1391]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1195vm-test-run-nixbot-gitlab> gitlab # [   13.346846] gitaly[1387]: time="2026-06-14T14:59:12.608Z" level=info msg="License database preloaded" duration_ms=2197 pid=13871196vm-test-run-nixbot-gitlab> gitlab # [   14.597928] dhcpcd[1232]: eth0: leased 10.0.2.15 for 86400 seconds1197vm-test-run-nixbot-gitlab> gitlab # [   14.600777] dhcpcd[1232]: eth0: adding route to 10.0.2.0/241198vm-test-run-nixbot-gitlab> gitlab # [   14.602910] dhcpcd[1232]: eth0: adding default route via 10.0.2.21199vm-test-run-nixbot-gitlab> gitlab # [   14.728313] systemd[1]: Started DHCP Client.1200vm-test-run-nixbot-gitlab> gitlab # [   14.732718] systemd[1]: Reached target Network is Online.1201vm-test-run-nixbot-gitlab> gitlab # [   40.961298] postgres[1730]: [1730] ERROR:  relation "feature_gates" does not exist at character 5231202vm-test-run-nixbot-gitlab> gitlab # [   40.967327] postgres[1730]: [1730] STATEMENT:  SELECT a.attname, format_type(a.atttypid, a.atttypmod),1203vm-test-run-nixbot-gitlab> gitlab # [   40.972827] postgres[1730]: 	       pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1204vm-test-run-nixbot-gitlab> gitlab # [   40.978781] postgres[1730]: 	       c.collname, col_description(a.attrelid, a.attnum) AS comment,1205vm-test-run-nixbot-gitlab> gitlab # [   40.984454] postgres[1730]: 	       attidentity AS identity,1206vm-test-run-nixbot-gitlab> gitlab # [   40.988464] postgres[1730]: 	       attgenerated as attgenerated1207vm-test-run-nixbot-gitlab> gitlab # [   40.992590] postgres[1730]: 	  FROM pg_attribute a1208vm-test-run-nixbot-gitlab> gitlab # [   40.995980] postgres[1730]: 	  LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1209vm-test-run-nixbot-gitlab> gitlab # [   41.001906] postgres[1730]: 	  LEFT JOIN pg_type t ON a.atttypid = t.oid1210vm-test-run-nixbot-gitlab> gitlab # [   41.006716] postgres[1730]: 	  LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1211vm-test-run-nixbot-gitlab> gitlab # [   41.012561] postgres[1730]: 	 WHERE a.attrelid = '"feature_gates"'::regclass1212vm-test-run-nixbot-gitlab> gitlab # [   41.016164] postgres[1730]: 	   AND a.attnum > 0 AND NOT a.attisdropped1213vm-test-run-nixbot-gitlab> gitlab # [   41.019587] postgres[1730]: 	 ORDER BY a.attnum1214vm-test-run-nixbot-gitlab> gitlab # [   41.022243] postgres[1730]:1215vm-test-run-nixbot-gitlab> gitlab # [   41.181337] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1391]: W, [2026-06-14T14:59:40.442808 #1391]  WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1216vm-test-run-nixbot-gitlab> gitlab # [   46.353911] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1785]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE:  extension "btree_gist" already exists, skipping1217vm-test-run-nixbot-gitlab> gitlab # [   46.355517] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1785]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE:  extension "pg_trgm" already exists, skipping1218vm-test-run-nixbot-gitlab> gitlab # [   49.187070] gitlab-workhorse[1290]: time="2026-06-14T14:59:48Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1219vm-test-run-nixbot-gitlab> gitlab # [   59.198219] gitlab-workhorse[1290]: time="2026-06-14T14:59:58Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1220vm-test-run-nixbot-gitlab> gitlab # [   61.037302] systemd[1]: Starting Logrotate Service...1221vm-test-run-nixbot-gitlab> gitlab # [   61.134587] systemd[1]: logrotate.service: Deactivated successfully.1222vm-test-run-nixbot-gitlab> gitlab # [   61.135390] systemd[1]: Finished Logrotate Service.1223vm-test-run-nixbot-gitlab> gitlab # [   63.027798] postgres[1869]: [1869] ERROR:  relation "table_without_models" does not exist1224vm-test-run-nixbot-gitlab> gitlab # [   63.032915] postgres[1869]: [1869] STATEMENT:  SELECT pg_get_serial_sequence('table_without_models', 'id')1225vm-test-run-nixbot-gitlab> gitlab # [   69.208491] gitlab-workhorse[1290]: time="2026-06-14T15:00:08Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1226vm-test-run-nixbot-gitlab> gitlab # [   77.276835] systemd[1]: Finished gitlab-db-config.service.1227vm-test-run-nixbot-gitlab> gitlab # [   77.281343] systemd[1]: Started gitlab-sidekiq.service.1228vm-test-run-nixbot-gitlab> gitlab # [   77.285118] systemd[1]: Starting gitlab.service...1229vm-test-run-nixbot-gitlab> gitlab # [   77.628883] bundle[2067]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1230vm-test-run-nixbot-gitlab> gitlab # [   77.631272] bundle[2067]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1231vm-test-run-nixbot-gitlab> gitlab # [   77.634608] bundle[2067]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1232vm-test-run-nixbot-gitlab> gitlab # [   79.218518] gitlab-workhorse[1290]: time="2026-06-14T15:00:18Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1233vm-test-run-nixbot-gitlab> gitlab # [   80.368749] sidekiq-cluster[2066]: {"severity":"INFO","time":"2026-06-14T15:00:19.613Z","message":"Starting cluster with 1 processes"}1234vm-test-run-nixbot-gitlab> gitlab # [   80.490233] bundle[2067]: {"timestamp":"2026-06-14T15:00:19.731Z","pid":2067,"message":"* Enabling systemd notification integration"}1235vm-test-run-nixbot-gitlab> gitlab # [   80.491374] bundle[2067]: {"timestamp":"2026-06-14T15:00:19.751Z","pid":2067,"message":"Puma starting in cluster mode..."}1236vm-test-run-nixbot-gitlab> gitlab # [   80.493143] bundle[2067]: {"timestamp":"2026-06-14T15:00:19.751Z","pid":2067,"message":"* Puma version: 7.2.0 (\"On The Corner\")"}1237vm-test-run-nixbot-gitlab> gitlab # [   80.494858] bundle[2067]: {"timestamp":"2026-06-14T15:00:19.751Z","pid":2067,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1238vm-test-run-nixbot-gitlab> gitlab # [   80.496378] bundle[2067]: {"timestamp":"2026-06-14T15:00:19.751Z","pid":2067,"message":"*  Min threads: 0"}1239vm-test-run-nixbot-gitlab> gitlab # [   80.498190] bundle[2067]: {"timestamp":"2026-06-14T15:00:19.751Z","pid":2067,"message":"*  Max threads: 4"}1240vm-test-run-nixbot-gitlab> gitlab # [   80.499588] bundle[2067]: {"timestamp":"2026-06-14T15:00:19.751Z","pid":2067,"message":"*  Environment: production"}1241vm-test-run-nixbot-gitlab> gitlab # [   80.501159] bundle[2067]: {"timestamp":"2026-06-14T15:00:19.752Z","pid":2067,"message":"*   Master PID: 2067"}1242vm-test-run-nixbot-gitlab> gitlab # [   80.502342] bundle[2067]: {"timestamp":"2026-06-14T15:00:19.752Z","pid":2067,"message":"*      Workers: 2"}1243vm-test-run-nixbot-gitlab> gitlab # [   80.504131] bundle[2067]: {"timestamp":"2026-06-14T15:00:19.752Z","pid":2067,"message":"*     Restarts: (✔) hot (✖) phased (✖) refork"}1244vm-test-run-nixbot-gitlab> gitlab # [   80.505520] bundle[2067]: {"timestamp":"2026-06-14T15:00:19.752Z","pid":2067,"message":"* Preloading application"}1245vm-test-run-nixbot-gitlab> gitlab # [   89.219943] gitlab-workhorse[1290]: time="2026-06-14T15:00:28Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1246vm-test-run-nixbot-gitlab> gitlab # [   99.231501] gitlab-workhorse[1290]: time="2026-06-14T15:00:38Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1247vm-test-run-nixbot-gitlab> gitlab # [  103.407620] bundle[2067]: W, [2026-06-14T15:00:42.668992 #2067]  WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1248vm-test-run-nixbot-gitlab> gitlab # [  106.241293] sidekiq-cluster[2097]: 2026-06-14T15:00:45.502Z pid=2097 tid=tyx INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2097, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>428425216}1249vm-test-run-nixbot-gitlab> gitlab # [  106.415750] sidekiq-cluster[2097]: W, [2026-06-14T15:00:45.677777 #2097]  WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1250vm-test-run-nixbot-gitlab> gitlab # [  106.455396] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:45.717Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1251vm-test-run-nixbot-gitlab> gitlab # [  108.380724] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.642Z","message":"GitLab reliable fetch activated!"}1252vm-test-run-nixbot-gitlab> gitlab # [  108.387785] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.642Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1253vm-test-run-nixbot-gitlab> gitlab # [  108.406503] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.668Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1254vm-test-run-nixbot-gitlab> gitlab # [  108.416184] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.672Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1255vm-test-run-nixbot-gitlab> gitlab # [  108.417085] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.676Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1256vm-test-run-nixbot-gitlab> gitlab # [  108.438360] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.700Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1257vm-test-run-nixbot-gitlab> gitlab # [  108.444718] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.706Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1258vm-test-run-nixbot-gitlab> gitlab # [  108.457665] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.719Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1259vm-test-run-nixbot-gitlab> gitlab # [  108.464327] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.723Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1260vm-test-run-nixbot-gitlab> gitlab # [  108.464926] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.727Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1261vm-test-run-nixbot-gitlab> gitlab # [  108.478049] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.740Z","message":"Cron Jobs - added job with name expire_build_artifacts_worker in the namespace default"}1262vm-test-run-nixbot-gitlab> gitlab # [  108.486484] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.748Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1263vm-test-run-nixbot-gitlab> gitlab # [  108.494901] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.757Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1264vm-test-run-nixbot-gitlab> gitlab # [  108.504509] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.766Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1265vm-test-run-nixbot-gitlab> gitlab # [  108.510326] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.772Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1266vm-test-run-nixbot-gitlab> gitlab # [  108.515712] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.777Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1267vm-test-run-nixbot-gitlab> gitlab # [  108.524952] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.787Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1268vm-test-run-nixbot-gitlab> gitlab # [  108.533687] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.795Z","message":"Cron Jobs - added job with name environments_auto_delete_cron_worker in the namespace default"}1269vm-test-run-nixbot-gitlab> gitlab # [  108.540815] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.802Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1270vm-test-run-nixbot-gitlab> gitlab # [  108.550652] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.812Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1271vm-test-run-nixbot-gitlab> gitlab # [  108.561465] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.823Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1272vm-test-run-nixbot-gitlab> gitlab # [  108.569518] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.831Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1273vm-test-run-nixbot-gitlab> gitlab # [  108.576712] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.838Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1274vm-test-run-nixbot-gitlab> gitlab # [  108.584215] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.846Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1275vm-test-run-nixbot-gitlab> gitlab # [  108.590183] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.852Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1276vm-test-run-nixbot-gitlab> gitlab # [  108.595983] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.858Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1277vm-test-run-nixbot-gitlab> gitlab # [  108.602105] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.864Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1278vm-test-run-nixbot-gitlab> gitlab # [  108.608177] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.870Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1279vm-test-run-nixbot-gitlab> gitlab # [  108.613517] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.875Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1280vm-test-run-nixbot-gitlab> gitlab # [  108.617829] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.880Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1281vm-test-run-nixbot-gitlab> gitlab # [  108.622550] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.884Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1282vm-test-run-nixbot-gitlab> gitlab # [  108.626935] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.889Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1283vm-test-run-nixbot-gitlab> gitlab # [  108.631681] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.893Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1284vm-test-run-nixbot-gitlab> gitlab # [  108.635526] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.897Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1285vm-test-run-nixbot-gitlab> gitlab # [  108.639465] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.901Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1286vm-test-run-nixbot-gitlab> gitlab # [  108.642841] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.905Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1287vm-test-run-nixbot-gitlab> gitlab # [  108.646092] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.908Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1288vm-test-run-nixbot-gitlab> gitlab # [  108.649335] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.911Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1289vm-test-run-nixbot-gitlab> gitlab # [  108.652565] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.914Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1290vm-test-run-nixbot-gitlab> gitlab # [  108.656113] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.918Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1291vm-test-run-nixbot-gitlab> gitlab # [  108.659537] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.921Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1292vm-test-run-nixbot-gitlab> gitlab # [  108.662590] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.924Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1293vm-test-run-nixbot-gitlab> gitlab # [  108.665741] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.928Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1294vm-test-run-nixbot-gitlab> gitlab # [  108.669403] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.931Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1295vm-test-run-nixbot-gitlab> gitlab # [  108.672378] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.934Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1296vm-test-run-nixbot-gitlab> gitlab # [  108.675696] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.938Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1297vm-test-run-nixbot-gitlab> gitlab # [  108.679302] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.941Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1298vm-test-run-nixbot-gitlab> gitlab # [  108.682988] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.945Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1299vm-test-run-nixbot-gitlab> gitlab # [  108.686377] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.948Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1300vm-test-run-nixbot-gitlab> gitlab # [  108.689295] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.951Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1301vm-test-run-nixbot-gitlab> gitlab # [  108.692746] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.955Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1302vm-test-run-nixbot-gitlab> gitlab # [  108.696851] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.959Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1303vm-test-run-nixbot-gitlab> gitlab # [  108.700536] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.962Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1304vm-test-run-nixbot-gitlab> gitlab # [  108.704303] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.966Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1305vm-test-run-nixbot-gitlab> gitlab # [  108.708148] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.970Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1306vm-test-run-nixbot-gitlab> gitlab # [  108.719601] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.981Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1307vm-test-run-nixbot-gitlab> gitlab # [  108.722941] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.985Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1308vm-test-run-nixbot-gitlab> gitlab # [  108.726762] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.989Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1309vm-test-run-nixbot-gitlab> gitlab # [  108.730199] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.992Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1310vm-test-run-nixbot-gitlab> gitlab # [  108.733534] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.995Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1311vm-test-run-nixbot-gitlab> gitlab # [  108.737639] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:47.999Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1312vm-test-run-nixbot-gitlab> gitlab # [  108.740971] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.003Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1313vm-test-run-nixbot-gitlab> gitlab # [  108.744466] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.006Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1314vm-test-run-nixbot-gitlab> gitlab # [  108.747781] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.010Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1315vm-test-run-nixbot-gitlab> gitlab # [  108.751077] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.013Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1316vm-test-run-nixbot-gitlab> gitlab # [  108.754878] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.017Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1317vm-test-run-nixbot-gitlab> gitlab # [  108.758925] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.021Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1318vm-test-run-nixbot-gitlab> gitlab # [  108.761843] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.024Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1319vm-test-run-nixbot-gitlab> gitlab # [  108.765928] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.028Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1320vm-test-run-nixbot-gitlab> gitlab # [  108.769369] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.031Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1321vm-test-run-nixbot-gitlab> gitlab # [  108.772563] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.034Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1322vm-test-run-nixbot-gitlab> gitlab # [  108.797631] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.059Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1323vm-test-run-nixbot-gitlab> gitlab # [  108.801923] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.064Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1324vm-test-run-nixbot-gitlab> gitlab # [  108.804975] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.067Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1325vm-test-run-nixbot-gitlab> gitlab # [  108.808580] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.070Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1326vm-test-run-nixbot-gitlab> gitlab # [  108.811774] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.074Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1327vm-test-run-nixbot-gitlab> gitlab # [  108.815638] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.077Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1328vm-test-run-nixbot-gitlab> gitlab # [  108.818910] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.081Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1329vm-test-run-nixbot-gitlab> gitlab # [  108.823266] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.085Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1330vm-test-run-nixbot-gitlab> gitlab # [  108.826301] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.088Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1331vm-test-run-nixbot-gitlab> gitlab # [  108.830143] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.092Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1332vm-test-run-nixbot-gitlab> gitlab # [  108.833083] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.095Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1333vm-test-run-nixbot-gitlab> gitlab # [  108.836745] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.099Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1334vm-test-run-nixbot-gitlab> gitlab # [  108.839874] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.102Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1335vm-test-run-nixbot-gitlab> gitlab # [  108.843781] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.106Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1336vm-test-run-nixbot-gitlab> gitlab # [  108.847788] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.110Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1337vm-test-run-nixbot-gitlab> gitlab # [  108.851654] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.113Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1338vm-test-run-nixbot-gitlab> gitlab # [  108.855153] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.117Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1339vm-test-run-nixbot-gitlab> gitlab # [  108.858564] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.120Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1340vm-test-run-nixbot-gitlab> gitlab # [  108.862270] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.124Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1341vm-test-run-nixbot-gitlab> gitlab # [  108.865929] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.128Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1342vm-test-run-nixbot-gitlab> gitlab # [  108.869312] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.131Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1343vm-test-run-nixbot-gitlab> gitlab # [  108.872717] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.135Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1344vm-test-run-nixbot-gitlab> gitlab # [  108.875911] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.138Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1345vm-test-run-nixbot-gitlab> gitlab # [  108.878725] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.141Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1346vm-test-run-nixbot-gitlab> gitlab # [  108.882453] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.144Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1347vm-test-run-nixbot-gitlab> gitlab # [  108.885569] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.147Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1348vm-test-run-nixbot-gitlab> gitlab # [  108.890102] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.152Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1349vm-test-run-nixbot-gitlab> gitlab # [  108.893715] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.156Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1350vm-test-run-nixbot-gitlab> gitlab # [  108.897071] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.159Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1351vm-test-run-nixbot-gitlab> gitlab # [  108.900038] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.162Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1352vm-test-run-nixbot-gitlab> gitlab # [  108.913851] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.176Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1353vm-test-run-nixbot-gitlab> gitlab # [  108.916272] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.178Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1354vm-test-run-nixbot-gitlab> gitlab # [  108.919307] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.181Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1355vm-test-run-nixbot-gitlab> gitlab # [  108.922979] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.185Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1356vm-test-run-nixbot-gitlab> gitlab # [  108.926995] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.189Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1357vm-test-run-nixbot-gitlab> gitlab # [  108.930601] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.192Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1358vm-test-run-nixbot-gitlab> gitlab # [  108.933815] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.196Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1359vm-test-run-nixbot-gitlab> gitlab # [  108.936438] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.198Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1360vm-test-run-nixbot-gitlab> gitlab # [  108.938689] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.200Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1361vm-test-run-nixbot-gitlab> gitlab # [  108.941942] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.204Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1362vm-test-run-nixbot-gitlab> gitlab # [  108.944270] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:00:48.206Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1363vm-test-run-nixbot-gitlab> gitlab # [  109.232991] gitlab-workhorse[1290]: time="2026-06-14T15:00:48Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1364vm-test-run-nixbot-gitlab> gitlab # [  119.236912] gitlab-workhorse[1290]: time="2026-06-14T15:00:58Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1365vm-test-run-nixbot-gitlab> gitlab # [  129.238886] gitlab-workhorse[1290]: time="2026-06-14T15:01:08Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1366vm-test-run-nixbot-gitlab> gitlab # [  130.091203] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:09.352Z","message":"Booted Rails 7.2.3 application in production environment"}1367vm-test-run-nixbot-gitlab> gitlab # [  130.099857] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:09.352Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1368vm-test-run-nixbot-gitlab> gitlab # [  130.108573] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:09.353Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1369vm-test-run-nixbot-gitlab> gitlab # [  130.116866] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:09.353Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1370vm-test-run-nixbot-gitlab> gitlab # [  130.125966] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:09.362Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1371vm-test-run-nixbot-gitlab> gitlab # [  130.970563] bundle[2067]: {"timestamp":"2026-06-14T15:01:10.231Z","pid":2067,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1372vm-test-run-nixbot-gitlab> gitlab # [  130.978851] bundle[2067]: {"timestamp":"2026-06-14T15:01:10.231Z","pid":2067,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1373vm-test-run-nixbot-gitlab> gitlab # [  130.986408] bundle[2067]: {"timestamp":"2026-06-14T15:01:10.231Z","pid":2067,"message":"! #\u003cThread:0x00007a6fe30a84a8 /nix/store/g5qraa51hgg5fl7n3da89ygf00qfv95x-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/g5qraa51hgg5fl7n3da89ygf00qfv95x-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1374vm-test-run-nixbot-gitlab> gitlab # [  131.003329] bundle[2067]: {"timestamp":"2026-06-14T15:01:10.232Z","pid":2067,"message":"! #\u003cThread:0x00007a6fe32d2c10 /nix/store/wg5r0zmy3rh1kga7z4z7mbs298p0wxli-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/wg5r0zmy3rh1kga7z4z7mbs298p0wxli-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1375vm-test-run-nixbot-gitlab> gitlab # [  131.012196] bundle[2067]: {"timestamp":"2026-06-14T15:01:10.232Z","pid":2067,"message":"Use Ctrl-C to stop"}1376vm-test-run-nixbot-gitlab> gitlab # [  131.014486] bundle[2067]: {"timestamp":"2026-06-14T15:01:10.232Z","pid":2067,"message":"Sending status to systemd every 1.0 sec"}1377vm-test-run-nixbot-gitlab> gitlab # [  131.374782] bundle[2067]: {"timestamp":"2026-06-14T15:01:10.636Z","pid":2067,"message":"- Worker 0 (PID: 2581) booted in 0.37s, phase: 0"}1378vm-test-run-nixbot-gitlab> gitlab # [  131.385154] bundle[2067]: {"timestamp":"2026-06-14T15:01:10.647Z","pid":2067,"message":"- Worker 1 (PID: 2585) booted in 0.36s, phase: 0"}1379vm-test-run-nixbot-gitlab> gitlab # [  131.391202] systemd[1]: Started gitlab.service.1380vm-test-run-nixbot-gitlab> gitlab # [  131.393968] systemd[1]: Reached target Common target for all GitLab services..1381vm-test-run-nixbot-gitlab> gitlab # [  131.398104] systemd[1]: Starting setup-gitlab.service...1382vm-test-run-nixbot-gitlab> gitlab # [  131.407436] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:10.669Z","message":"Cleaning working queues"}1383vm-test-run-nixbot-gitlab> gitlab # [  131.412881] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:10.674Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"06a7a7f74958f149d00d32bb","created_at":"2026-06-14T15:00:15.985Z","trace_propagation_headers":{"sentry-trace":"b09f5c49154244b8a03de46585ac980c-f8dbf31f08aa459e","baggage":"sentry-trace_id=b09f5c49154244b8a03de46585ac980c,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"ac30a0c558ad659bd4d42984ec37200e","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-14T15:00:15.998Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-06a7a7f74958f149d00d32bb: start","job_status":"start","queue_duration_s":54.67586,"scheduling_latency_s":54.675943}1384vm-test-run-nixbot-gitlab> gitlab # [  131.433247] setup-gitlab-start[2618]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1385vm-test-run-nixbot-gitlab> gitlab # [  131.601796] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:10.863Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"06a7a7f74958f149d00d32bb","created_at":"2026-06-14T15:00:15.985Z","trace_propagation_headers":{"sentry-trace":"b09f5c49154244b8a03de46585ac980c-f8dbf31f08aa459e","baggage":"sentry-trace_id=b09f5c49154244b8a03de46585ac980c,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"ac30a0c558ad659bd4d42984ec37200e","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-14T15:00:15.998Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-06a7a7f74958f149d00d32bb: done: 0.188721 sec","job_status":"done","queue_duration_s":54.67586,"scheduling_latency_s":54.675943,"redis_calls":24,"redis_duration_s":0.008002,"redis_read_bytes":912,"redis_write_bytes":2622,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.003461,"redis_feature_flag_read_bytes":902,"redis_feature_flag_write_bytes":1375,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002914,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001627,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.082811,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.090229904,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.188721,"completed_at":"2026-06-14T15:01:10.862Z","load_balancing_strategy":"primary","db_duration_s":0.05432,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1386vm-test-run-nixbot-gitlab> gitlab # [  133.619420] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:01:12 +0000] "GET /users/sign_in HTTP/1.1" 200 10624 "" "curl/8.20.0" 21731387vm-test-run-nixbot-gitlab> gitlab # [  133.633158] setup-gitlab-start[2644]: ++ curl -fs -X POST -H 'Content-Type: application/json' -d '{"grant_type":"password","username":"root","password":"notproduction"}' http://gitlab/oauth/token1388vm-test-run-nixbot-gitlab> gitlab # [  133.642715] setup-gitlab-start[2645]: ++ jq -r .access_token1389vm-test-run-nixbot-gitlab> gitlab # [  134.383872] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:01:13 +0000] "POST /oauth/token HTTP/1.1" 200 244 "" "curl/8.20.0" 7321390vm-test-run-nixbot-gitlab> gitlab # [  134.395338] setup-gitlab-start[2618]: + BEARER=9857042119ca9ef6839634ae0326d9ed1fcf9afc058fdf9a40aa22e4e0a4d5661391vm-test-run-nixbot-gitlab> gitlab # [  134.396373] setup-gitlab-start[2618]: + AUTH='Authorization: Bearer 9857042119ca9ef6839634ae0326d9ed1fcf9afc058fdf9a40aa22e4e0a4d566'1392vm-test-run-nixbot-gitlab> gitlab # [  134.399054] setup-gitlab-start[2618]: + curl -fs -X PUT -H 'Authorization: Bearer 9857042119ca9ef6839634ae0326d9ed1fcf9afc058fdf9a40aa22e4e0a4d566' 'http://gitlab/api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true'1393vm-test-run-nixbot-gitlab> gitlab # [  135.770420] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:01:15 +0000] "PUT /api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true HTTP/1.1" 200 17133 "" "curl/8.20.0" 13541394vm-test-run-nixbot-gitlab> gitlab # [  135.776982] setup-gitlab-start[2618]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner Rails.cache.clear1395vm-test-run-nixbot-gitlab> gitlab # [  135.793264] runuser[2669]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1396vm-test-run-nixbot-gitlab> gitlab # [  136.120615] setup-gitlab-start[2670]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1397vm-test-run-nixbot-gitlab> gitlab # [  136.121318] setup-gitlab-start[2670]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1398vm-test-run-nixbot-gitlab> gitlab # [  136.124429] setup-gitlab-start[2670]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1399vm-test-run-nixbot-gitlab> gitlab # [  136.684344] setup-gitlab-start[2670]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1400vm-test-run-nixbot-gitlab> gitlab # [  136.685872] setup-gitlab-start[2670]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1401vm-test-run-nixbot-gitlab> gitlab # [  136.689515] setup-gitlab-start[2670]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1402vm-test-run-nixbot-gitlab> gitlab # [  143.975905] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:23.237Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"2c627c63c762cabd013009d2","created_at":"2026-06-14T15:01:23.198Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5fb854202cd2daebcd930f58e5f1bb5d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-14T15:01:23.232Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-2c627c63c762cabd013009d2: start","job_status":"start","queue_duration_s":0.004353,"scheduling_latency_s":0.004423}1403vm-test-run-nixbot-gitlab> gitlab # [  144.083590] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:23.345Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1404vm-test-run-nixbot-gitlab> gitlab # [  144.133527] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:23.394Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"2c627c63c762cabd013009d2","created_at":"2026-06-14T15:01:23.198Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5fb854202cd2daebcd930f58e5f1bb5d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-14T15:01:23.232Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-2c627c63c762cabd013009d2: done: 0.15766 sec","job_status":"done","queue_duration_s":0.004353,"scheduling_latency_s":0.004423,"redis_calls":10,"redis_duration_s":0.026785,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003083,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.021015,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002687,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030037,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001513879,"duration_s":0.15766,"completed_at":"2026-06-14T15:01:23.394Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002616,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1405vm-test-run-nixbot-gitlab> gitlab # [  144.224418] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:23.401Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Gitlab::JiraImport::StuckJiraImportJobsWorker","jid":"242544b72ef7a7264a7cdbd8","created_at":"2026-06-14T15:01:23.323Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"494ecbdf8ee6eee6802dc6374b6ae1a4","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b9cf6c9b6c2f8fad80e08693d59af787bb65ad6ffe24ec5d0e1a669517566127","size_limiter":"validated","enqueued_at":"2026-06-14T15:01:23.348Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Gitlab::JiraImport::StuckJiraImportJobsWorker JID-242544b72ef7a7264a7cdbd8: start","job_status":"start","queue_duration_s":0.052592,"scheduling_latency_s":0.052646}1406vm-test-run-nixbot-gitlab> gitlab # [  144.284213] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:23.546Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Gitlab::JiraImport::StuckJiraImportJobsWorker","jid":"242544b72ef7a7264a7cdbd8","created_at":"2026-06-14T15:01:23.323Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"494ecbdf8ee6eee6802dc6374b6ae1a4","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b9cf6c9b6c2f8fad80e08693d59af787bb65ad6ffe24ec5d0e1a669517566127","size_limiter":"validated","enqueued_at":"2026-06-14T15:01:23.348Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Gitlab::JiraImport::StuckJiraImportJobsWorker JID-242544b72ef7a7264a7cdbd8: done: 0.145339 sec","job_status":"done","queue_duration_s":0.052592,"scheduling_latency_s":0.052646,"redis_calls":10,"redis_duration_s":0.011009,"redis_read_bytes":9,"redis_write_bytes":1187,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002618,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004964,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003427,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027049,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00161501,"duration_s":0.145339,"completed_at":"2026-06-14T15:01:23.546Z","load_balancing_strategy":"primary","db_duration_s":0.017402,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1407vm-test-run-nixbot-gitlab> gitlab # [  144.323994] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:23.585Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"ba3912f4c228f4224d36aab6","created_at":"2026-06-14T15:01:23.386Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f462042c6097d0550f0548cf50836e78","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-14T15:01:23.491Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-ba3912f4c228f4224d36aab6: start","job_status":"start","queue_duration_s":0.094195,"scheduling_latency_s":0.094227}1408vm-test-run-nixbot-gitlab> gitlab # [  145.058785] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:24.319Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"ba3912f4c228f4224d36aab6","created_at":"2026-06-14T15:01:23.386Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f462042c6097d0550f0548cf50836e78","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-14T15:01:23.491Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-ba3912f4c228f4224d36aab6: done: 0.733678 sec","job_status":"done","queue_duration_s":0.094195,"scheduling_latency_s":0.094227,"redis_calls":574,"redis_duration_s":0.37060299999999996,"redis_read_bytes":573,"redis_write_bytes":49866,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000758,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002855,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.36699,"redis_shared_state_read_bytes":567,"redis_shared_state_write_bytes":49017,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.405953,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000805689,"duration_s":0.733678,"completed_at":"2026-06-14T15:01:24.319Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000807,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1409vm-test-run-nixbot-gitlab> gitlab # [  145.144377] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:24.406Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"c3437b0b2dea584d9d43fc73","created_at":"2026-06-14T15:01:23.500Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"19861d2c2aa4cdd99cf53c23f086e10d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-14T15:01:23.515Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-c3437b0b2dea584d9d43fc73: start","job_status":"start","queue_duration_s":0.890633,"scheduling_latency_s":0.890703}1410vm-test-run-nixbot-gitlab> gitlab # [  145.550821] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:24.812Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"c3437b0b2dea584d9d43fc73","created_at":"2026-06-14T15:01:23.500Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"19861d2c2aa4cdd99cf53c23f086e10d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-14T15:01:23.515Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-c3437b0b2dea584d9d43fc73: done: 0.406889 sec","job_status":"done","queue_duration_s":0.890633,"scheduling_latency_s":0.890703,"redis_calls":14,"redis_duration_s":0.008258999999999999,"redis_read_bytes":13,"redis_write_bytes":1641,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002255,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":576,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003692,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002312,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":125,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":125,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.054,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":123,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":80,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.052,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.306789,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001297651,"exclusive_lock_hold_duration_s":0.361043373,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.406889,"completed_at":"2026-06-14T15:01:24.812Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.09046,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1411vm-test-run-nixbot-gitlab> gitlab # [  145.651878] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:24.913Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"9e6788f20b5ad04b6e257bf8","created_at":"2026-06-14T15:01:23.533Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"594e45884ee6efbff13b27eefaeb2bf3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-14T15:01:23.553Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-9e6788f20b5ad04b6e257bf8: start","job_status":"start","queue_duration_s":1.359698,"scheduling_latency_s":1.359736}1412vm-test-run-nixbot-gitlab> gitlab # [  145.709924] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:24.971Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"9e6788f20b5ad04b6e257bf8","created_at":"2026-06-14T15:01:23.533Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"594e45884ee6efbff13b27eefaeb2bf3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-14T15:01:23.553Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-9e6788f20b5ad04b6e257bf8: done: 0.058004 sec","job_status":"done","queue_duration_s":1.359698,"scheduling_latency_s":1.359736,"redis_calls":12,"redis_duration_s":0.007281,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001204,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003421,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002656,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032122,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001259937,"exclusive_lock_hold_duration_s":0.015908725999999998,"duration_s":0.058004,"completed_at":"2026-06-14T15:01:24.971Z","load_balancing_strategy":"primary","db_duration_s":0.010004,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1413vm-test-run-nixbot-gitlab> gitlab # [  145.778692] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.040Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"6051cc24bf05cf733b2dd9eb","created_at":"2026-06-14T15:01:23.646Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a784caf16c77df69951a6d8ddf79bf8d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-14T15:01:23.655Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-6051cc24bf05cf733b2dd9eb: start","job_status":"start","queue_duration_s":1.384544,"scheduling_latency_s":1.384598}1414vm-test-run-nixbot-gitlab> gitlab # [  145.826366] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.088Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1415vm-test-run-nixbot-gitlab> gitlab # [  145.846261] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.106Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"6051cc24bf05cf733b2dd9eb","created_at":"2026-06-14T15:01:23.646Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a784caf16c77df69951a6d8ddf79bf8d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-14T15:01:23.655Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-6051cc24bf05cf733b2dd9eb: done: 0.065238 sec","job_status":"done","queue_duration_s":1.384544,"scheduling_latency_s":1.384598,"redis_calls":10,"redis_duration_s":0.006296,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001308,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003847,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001141,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019323,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000975263,"duration_s":0.065238,"completed_at":"2026-06-14T15:01:25.105Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001151,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1416vm-test-run-nixbot-gitlab> gitlab # [  145.848397] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.110Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"1de88e4d7f79f55506be4d4e","created_at":"2026-06-14T15:01:23.665Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-14T15:01:23.672Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-1de88e4d7f79f55506be4d4e: start","job_status":"start","queue_duration_s":1.437253,"scheduling_latency_s":1.437304}1417vm-test-run-nixbot-gitlab> gitlab # [  145.952358] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.214Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"1de88e4d7f79f55506be4d4e","created_at":"2026-06-14T15:01:23.665Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-14T15:01:23.672Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-1de88e4d7f79f55506be4d4e: done: 0.104112 sec","job_status":"done","queue_duration_s":1.437253,"scheduling_latency_s":1.437304,"redis_calls":71,"redis_duration_s":0.018455,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002491,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001004,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.01396,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.074661,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000515149,"duration_s":0.104112,"completed_at":"2026-06-14T15:01:25.214Z","load_balancing_strategy":"primary","db_duration_s":0.001685,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1418vm-test-run-nixbot-gitlab> gitlab # [  146.021251] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.283Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"44eb8af6d766ba80b28001d8","created_at":"2026-06-14T15:01:23.684Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"759b6f209581bc9dbccf33888f25d576","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-14T15:01:23.692Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-44eb8af6d766ba80b28001d8: start","job_status":"start","queue_duration_s":1.590339,"scheduling_latency_s":1.590375}1419vm-test-run-nixbot-gitlab> gitlab # [  146.044837] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.297Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1420vm-test-run-nixbot-gitlab> gitlab # [  146.046269] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.304Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"44eb8af6d766ba80b28001d8","created_at":"2026-06-14T15:01:23.684Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"759b6f209581bc9dbccf33888f25d576","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-14T15:01:23.692Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-44eb8af6d766ba80b28001d8: done: 0.021136 sec","job_status":"done","queue_duration_s":1.590339,"scheduling_latency_s":1.590375,"redis_calls":10,"redis_duration_s":0.004037,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000898,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002397,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000742,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010783,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000534425,"duration_s":0.021136,"completed_at":"2026-06-14T15:01:25.304Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000887,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1421vm-test-run-nixbot-gitlab> gitlab # [  146.048392] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.306Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"31cc940b1542f206fdc0cc73","created_at":"2026-06-14T15:01:23.705Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bada6da115ed4a70280fca2a2c943692","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-14T15:01:23.712Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-31cc940b1542f206fdc0cc73: start","job_status":"start","queue_duration_s":1.593272,"scheduling_latency_s":1.593292}1422vm-test-run-nixbot-gitlab> gitlab # [  146.218789] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.478Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"31cc940b1542f206fdc0cc73","created_at":"2026-06-14T15:01:23.705Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bada6da115ed4a70280fca2a2c943692","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-14T15:01:23.712Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-31cc940b1542f206fdc0cc73: done: 0.172739 sec","job_status":"done","queue_duration_s":1.593272,"scheduling_latency_s":1.593292,"redis_calls":16,"redis_duration_s":0.007712,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.003272,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003463,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000977,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043344,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001565841,"duration_s":0.172739,"completed_at":"2026-06-14T15:01:25.478Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.019488,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1423vm-test-run-nixbot-gitlab> gitlab # [  146.272844] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.483Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"0b6b8f3cf65b3763919295df","created_at":"2026-06-14T15:01:23.721Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e5feb8aac2fccac6346fadc67b45ed3a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-14T15:01:23.729Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-0b6b8f3cf65b3763919295df: start","job_status":"start","queue_duration_s":1.754046,"scheduling_latency_s":1.754198}1424vm-test-run-nixbot-gitlab> gitlab # [  146.283229] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.545Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1425vm-test-run-nixbot-gitlab> gitlab # [  146.291629] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.553Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"0b6b8f3cf65b3763919295df","created_at":"2026-06-14T15:01:23.721Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e5feb8aac2fccac6346fadc67b45ed3a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-14T15:01:23.729Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-0b6b8f3cf65b3763919295df: done: 0.070277 sec","job_status":"done","queue_duration_s":1.754046,"scheduling_latency_s":1.754198,"redis_calls":10,"redis_duration_s":0.004146,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000909,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002341,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000896,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012705,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001057677,"duration_s":0.070277,"completed_at":"2026-06-14T15:01:25.553Z","load_balancing_strategy":"primary","db_duration_s":0.000899,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1426vm-test-run-nixbot-gitlab> gitlab # [  146.393886] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.555Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"16926e074ba3cf20df34ade2","created_at":"2026-06-14T15:01:23.737Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1a15c215e73e398881468e473d3e6d80","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-14T15:01:23.744Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-16926e074ba3cf20df34ade2: start","job_status":"start","queue_duration_s":1.810153,"scheduling_latency_s":1.810179}1427vm-test-run-nixbot-gitlab> gitlab # [  146.395532] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.579Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"16926e074ba3cf20df34ade2","created_at":"2026-06-14T15:01:23.737Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1a15c215e73e398881468e473d3e6d80","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-14T15:01:23.744Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-16926e074ba3cf20df34ade2: done: 0.024074 sec","job_status":"done","queue_duration_s":1.810153,"scheduling_latency_s":1.810179,"redis_calls":10,"redis_duration_s":0.002451,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000548,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001461,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000442,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014879,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00039586,"duration_s":0.024074,"completed_at":"2026-06-14T15:01:25.579Z","load_balancing_strategy":"primary","db_duration_s":0.002467,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1428vm-test-run-nixbot-gitlab> gitlab # [  146.413293] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.580Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"8b3eca4ba012531b133632fb","created_at":"2026-06-14T15:01:23.758Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4b753ac11823a71f93cfd89bbdf96feb","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-14T15:01:23.765Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-8b3eca4ba012531b133632fb: start","job_status":"start","queue_duration_s":1.814651,"scheduling_latency_s":1.814673}1429vm-test-run-nixbot-gitlab> gitlab # [  146.487569] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.602Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"8b3eca4ba012531b133632fb","created_at":"2026-06-14T15:01:23.758Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4b753ac11823a71f93cfd89bbdf96feb","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-14T15:01:23.765Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-8b3eca4ba012531b133632fb: done: 0.021729 sec","job_status":"done","queue_duration_s":1.814651,"scheduling_latency_s":1.814673,"redis_calls":10,"redis_duration_s":0.002996,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000702,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001723,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000571,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011234,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000501461,"duration_s":0.021729,"completed_at":"2026-06-14T15:01:25.602Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003495,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1430vm-test-run-nixbot-gitlab> gitlab # [  146.501952] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.604Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"aa5a55f6722a4daf9b327319","created_at":"2026-06-14T15:01:23.790Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b9b890b1f88389ded1e7e55e6de07fb5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-14T15:01:23.798Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-aa5a55f6722a4daf9b327319: start","job_status":"start","queue_duration_s":1.805452,"scheduling_latency_s":1.805474}1431vm-test-run-nixbot-gitlab> gitlab # [  146.523440] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.637Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"aa5a55f6722a4daf9b327319","created_at":"2026-06-14T15:01:23.790Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b9b890b1f88389ded1e7e55e6de07fb5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-14T15:01:23.798Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-aa5a55f6722a4daf9b327319: done: 0.032613 sec","job_status":"done","queue_duration_s":1.805452,"scheduling_latency_s":1.805474,"redis_calls":19,"redis_duration_s":0.007573,"redis_read_bytes":22,"redis_write_bytes":4785,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001631,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000574,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.00399,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001378,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018562,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000502578,"duration_s":0.032613,"completed_at":"2026-06-14T15:01:25.636Z","load_balancing_strategy":"primary","db_duration_s":0.00285,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1432vm-test-run-nixbot-gitlab> gitlab # [  146.632455] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.638Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"7b95621f537d247fdc5ddda0","created_at":"2026-06-14T15:01:23.808Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f287b735c9bb2bbe7e89b49222d4c24f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-14T15:01:23.815Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-7b95621f537d247fdc5ddda0: start","job_status":"start","queue_duration_s":1.823466,"scheduling_latency_s":1.823487}1433vm-test-run-nixbot-gitlab> gitlab # [  146.633861] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.661Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"7b95621f537d247fdc5ddda0","created_at":"2026-06-14T15:01:23.808Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f287b735c9bb2bbe7e89b49222d4c24f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-14T15:01:23.815Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-7b95621f537d247fdc5ddda0: done: 0.022275 sec","job_status":"done","queue_duration_s":1.823466,"scheduling_latency_s":1.823487,"redis_calls":12,"redis_duration_s":0.0044080000000000005,"redis_read_bytes":11,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001673,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002098,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000637,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011819,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000438324,"duration_s":0.022275,"completed_at":"2026-06-14T15:01:25.661Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001353,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1434vm-test-run-nixbot-gitlab> gitlab # [  146.714188] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.662Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"af83041bc44f8fb055fbd3fd","created_at":"2026-06-14T15:01:23.838Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"271432d39320a869962e9739a010b3fe","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-14T15:01:23.846Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-af83041bc44f8fb055fbd3fd: start","job_status":"start","queue_duration_s":1.816011,"scheduling_latency_s":1.816032}1435vm-test-run-nixbot-gitlab> gitlab # [  146.716259] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.691Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"af83041bc44f8fb055fbd3fd","created_at":"2026-06-14T15:01:23.838Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"271432d39320a869962e9739a010b3fe","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-14T15:01:23.846Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-af83041bc44f8fb055fbd3fd: done: 0.029131 sec","job_status":"done","queue_duration_s":1.816011,"scheduling_latency_s":1.816032,"redis_calls":10,"redis_duration_s":0.0024,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000436,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00153,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000434,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013577,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000406477,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.029131,"completed_at":"2026-06-14T15:01:25.691Z","load_balancing_strategy":"primary","db_duration_s":0.008928,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1436vm-test-run-nixbot-gitlab> gitlab # [  146.832269] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.693Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"662811daf4888d1e893c71ba","created_at":"2026-06-14T15:01:23.856Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fc8ce97e1d04d9a13c63457a4f194738","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-14T15:01:23.864Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-662811daf4888d1e893c71ba: start","job_status":"start","queue_duration_s":1.828543,"scheduling_latency_s":1.828564}1437vm-test-run-nixbot-gitlab> gitlab # [  146.833842] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.704Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1438vm-test-run-nixbot-gitlab> gitlab # [  146.835267] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.710Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"662811daf4888d1e893c71ba","created_at":"2026-06-14T15:01:23.856Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fc8ce97e1d04d9a13c63457a4f194738","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-14T15:01:23.864Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-662811daf4888d1e893c71ba: done: 0.017396 sec","job_status":"done","queue_duration_s":1.828543,"scheduling_latency_s":1.828564,"redis_calls":10,"redis_duration_s":0.002145,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000416,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001257,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000472,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010225,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032099,"duration_s":0.017396,"completed_at":"2026-06-14T15:01:25.710Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000398,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1439vm-test-run-nixbot-gitlab> gitlab # [  146.924209] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.712Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"3a3c276a7b274f09e3ad7eb3","created_at":"2026-06-14T15:01:23.874Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ea8c5823654211945fc4ac6c5ce2a406","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-14T15:01:23.881Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-3a3c276a7b274f09e3ad7eb3: start","job_status":"start","queue_duration_s":1.830996,"scheduling_latency_s":1.831025}1440vm-test-run-nixbot-gitlab> gitlab # [  146.925423] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.728Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"3a3c276a7b274f09e3ad7eb3","created_at":"2026-06-14T15:01:23.874Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ea8c5823654211945fc4ac6c5ce2a406","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-14T15:01:23.881Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-3a3c276a7b274f09e3ad7eb3: done: 0.016738 sec","job_status":"done","queue_duration_s":1.830996,"scheduling_latency_s":1.831025,"redis_calls":10,"redis_duration_s":0.002254,"redis_read_bytes":9,"redis_write_bytes":1215,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000473,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001323,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000458,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01003,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000341104,"duration_s":0.016738,"completed_at":"2026-06-14T15:01:25.728Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00035,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1441vm-test-run-nixbot-gitlab> gitlab # [  147.003190] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.730Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"30590c8884c80a326d67bb21","created_at":"2026-06-14T15:01:23.889Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"06f01a34a62e331781aee1b4635a756b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-14T15:01:23.896Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-30590c8884c80a326d67bb21: start","job_status":"start","queue_duration_s":1.833724,"scheduling_latency_s":1.833744}1442vm-test-run-nixbot-gitlab> gitlab # [  147.004865] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.765Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"30590c8884c80a326d67bb21","created_at":"2026-06-14T15:01:23.889Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"06f01a34a62e331781aee1b4635a756b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-14T15:01:23.896Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-30590c8884c80a326d67bb21: done: 0.035341 sec","job_status":"done","queue_duration_s":1.833724,"scheduling_latency_s":1.833744,"redis_calls":12,"redis_duration_s":0.002874,"redis_read_bytes":11,"redis_write_bytes":1612,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000402,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":347,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001454,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001018,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":7,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017685,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000434692,"exclusive_lock_hold_duration_s":0.017796954,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.035341,"completed_at":"2026-06-14T15:01:25.765Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011142,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1443vm-test-run-nixbot-gitlab> gitlab # [  147.075063] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.766Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"b1dcebeb5b5df3265d00c868","created_at":"2026-06-14T15:01:23.903Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d534a38e823329f72ada11e0f1cd6c39","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-14T15:01:23.909Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-b1dcebeb5b5df3265d00c868: start","job_status":"start","queue_duration_s":1.857043,"scheduling_latency_s":1.857064}1444vm-test-run-nixbot-gitlab> gitlab # [  147.079313] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.787Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"b1dcebeb5b5df3265d00c868","created_at":"2026-06-14T15:01:23.903Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d534a38e823329f72ada11e0f1cd6c39","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-14T15:01:23.909Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-b1dcebeb5b5df3265d00c868: done: 0.021082 sec","job_status":"done","queue_duration_s":1.857043,"scheduling_latency_s":1.857064,"redis_calls":12,"redis_duration_s":0.002929,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000543,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001327,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001059,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011766,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000445866,"exclusive_lock_hold_duration_s":0.004066996,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.021082,"completed_at":"2026-06-14T15:01:25.787Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002277,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1445vm-test-run-nixbot-gitlab> gitlab # [  147.214347] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.789Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"3a5eb9254feb5ad8882678fe","created_at":"2026-06-14T15:01:23.926Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3505d1d02b809f92d8b7722f078d6692","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-14T15:01:23.937Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-3a5eb9254feb5ad8882678fe: start","job_status":"start","queue_duration_s":1.851719,"scheduling_latency_s":1.851739}1446vm-test-run-nixbot-gitlab> gitlab # [  147.215493] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.815Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"3a5eb9254feb5ad8882678fe","created_at":"2026-06-14T15:01:23.926Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3505d1d02b809f92d8b7722f078d6692","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-14T15:01:23.937Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-3a5eb9254feb5ad8882678fe: done: 0.026205 sec","job_status":"done","queue_duration_s":1.851719,"scheduling_latency_s":1.851739,"redis_calls":13,"redis_duration_s":0.003101,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000468,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00161,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001023,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016048,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.008114719000000001,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.026205,"completed_at":"2026-06-14T15:01:25.815Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002851,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1447vm-test-run-nixbot-gitlab> gitlab # [  147.242504] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.817Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"db77449cf7abf001cd0fcfd1","created_at":"2026-06-14T15:01:23.951Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a6828abd8a822ca46a7fe452dc4c2489","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-14T15:01:23.962Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-db77449cf7abf001cd0fcfd1: start","job_status":"start","queue_duration_s":1.854503,"scheduling_latency_s":1.854523}1448vm-test-run-nixbot-gitlab> gitlab # [  147.263856] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.827Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1449vm-test-run-nixbot-gitlab> gitlab # [  147.268773] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.833Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"db77449cf7abf001cd0fcfd1","created_at":"2026-06-14T15:01:23.951Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a6828abd8a822ca46a7fe452dc4c2489","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-14T15:01:23.962Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-db77449cf7abf001cd0fcfd1: done: 0.016591 sec","job_status":"done","queue_duration_s":1.854503,"scheduling_latency_s":1.854523,"redis_calls":10,"redis_duration_s":0.002261,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00042,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001298,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000543,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009329,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000345016,"duration_s":0.016591,"completed_at":"2026-06-14T15:01:25.833Z","load_balancing_strategy":"primary","db_duration_s":0.000355,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1450vm-test-run-nixbot-gitlab> gitlab # [  147.360044] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.835Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"35d706360ab9ca4732bf7f93","created_at":"2026-06-14T15:01:23.972Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dcaf88d1d9b458a5e1294b884f19cf33","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-14T15:01:23.979Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-35d706360ab9ca4732bf7f93: start","job_status":"start","queue_duration_s":1.855375,"scheduling_latency_s":1.855396}1451vm-test-run-nixbot-gitlab> gitlab # [  147.361190] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.865Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"35d706360ab9ca4732bf7f93","created_at":"2026-06-14T15:01:23.972Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dcaf88d1d9b458a5e1294b884f19cf33","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-14T15:01:23.979Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-35d706360ab9ca4732bf7f93: done: 0.030329 sec","job_status":"done","queue_duration_s":1.855375,"scheduling_latency_s":1.855396,"redis_calls":12,"redis_duration_s":0.002951,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00047,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001379,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001102,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01549,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000474641,"exclusive_lock_hold_duration_s":0.009137194,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.030329,"completed_at":"2026-06-14T15:01:25.865Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008338,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1452vm-test-run-nixbot-gitlab> gitlab # [  147.460853] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.866Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e07e7f78f5173bbf893593f","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e07e7f78f5173bbf893593f: start","job_status":"start","queue_duration_s":0.664356,"scheduling_latency_s":0.664378}1453vm-test-run-nixbot-gitlab> gitlab # [  147.462246] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.887Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e07e7f78f5173bbf893593f","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e07e7f78f5173bbf893593f: done: 0.020864 sec","job_status":"done","queue_duration_s":0.664356,"scheduling_latency_s":0.664378,"redis_calls":16,"redis_duration_s":0.003926,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000429,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001136,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002361,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012324,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000489727,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020864,"completed_at":"2026-06-14T15:01:25.887Z","load_balancing_strategy":"primary","db_duration_s":0.000321,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1454vm-test-run-nixbot-gitlab> gitlab # [  147.556486] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.889Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c20a93653b462530f0baa22a","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c20a93653b462530f0baa22a: start","job_status":"start","queue_duration_s":0.686474,"scheduling_latency_s":0.686495}1455vm-test-run-nixbot-gitlab> gitlab # [  147.557883] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.904Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c20a93653b462530f0baa22a","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c20a93653b462530f0baa22a: done: 0.015584 sec","job_status":"done","queue_duration_s":0.686474,"scheduling_latency_s":0.686495,"redis_calls":13,"redis_duration_s":0.0029119999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000871,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002041,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009838,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015584,"completed_at":"2026-06-14T15:01:25.904Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1456vm-test-run-nixbot-gitlab> gitlab # [  147.644122] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.905Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30f0a663d7d06c601ec54388","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30f0a663d7d06c601ec54388: start","job_status":"start","queue_duration_s":0.703365,"scheduling_latency_s":0.703385}1457vm-test-run-nixbot-gitlab> gitlab # [  147.644930] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.921Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30f0a663d7d06c601ec54388","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30f0a663d7d06c601ec54388: done: 0.015572 sec","job_status":"done","queue_duration_s":0.703365,"scheduling_latency_s":0.703385,"redis_calls":13,"redis_duration_s":0.002837,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000847,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00199,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009792,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015572,"completed_at":"2026-06-14T15:01:25.921Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1458vm-test-run-nixbot-gitlab> gitlab # [  147.672083] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.922Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6f32069fb793a47d0a509394","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6f32069fb793a47d0a509394: start","job_status":"start","queue_duration_s":0.720335,"scheduling_latency_s":0.720356}1459vm-test-run-nixbot-gitlab> gitlab # [  147.692124] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.938Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6f32069fb793a47d0a509394","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6f32069fb793a47d0a509394: done: 0.015337 sec","job_status":"done","queue_duration_s":0.720335,"scheduling_latency_s":0.720356,"redis_calls":13,"redis_duration_s":0.0029779999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000946,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002032,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009671,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015337,"completed_at":"2026-06-14T15:01:25.938Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1460vm-test-run-nixbot-gitlab> gitlab # [  147.789087] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.939Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f0e5ba1ccfe0bb8c3a94f9af","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f0e5ba1ccfe0bb8c3a94f9af: start","job_status":"start","queue_duration_s":0.736919,"scheduling_latency_s":0.736982}1461vm-test-run-nixbot-gitlab> gitlab # [  147.789858] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.955Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f0e5ba1ccfe0bb8c3a94f9af","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f0e5ba1ccfe0bb8c3a94f9af: done: 0.015586 sec","job_status":"done","queue_duration_s":0.736919,"scheduling_latency_s":0.736982,"redis_calls":13,"redis_duration_s":0.0028770000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000938,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001939,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009698,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015586,"completed_at":"2026-06-14T15:01:25.954Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1462vm-test-run-nixbot-gitlab> gitlab # [  147.861219] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.956Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2186088e5e8aec0468ac5add","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2186088e5e8aec0468ac5add: start","job_status":"start","queue_duration_s":0.754335,"scheduling_latency_s":0.754374}1463vm-test-run-nixbot-gitlab> gitlab # [  147.862736] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.974Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2186088e5e8aec0468ac5add","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2186088e5e8aec0468ac5add: done: 0.017408 sec","job_status":"done","queue_duration_s":0.754335,"scheduling_latency_s":0.754374,"redis_calls":13,"redis_duration_s":0.002819,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00088,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001939,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010859,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017408,"completed_at":"2026-06-14T15:01:25.974Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1464vm-test-run-nixbot-gitlab> gitlab # [  147.969444] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.979Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e00e854eb7fa621be8ca34bf","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e00e854eb7fa621be8ca34bf: start","job_status":"start","queue_duration_s":0.776576,"scheduling_latency_s":0.776598}1465vm-test-run-nixbot-gitlab> gitlab # [  147.970825] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.997Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e00e854eb7fa621be8ca34bf","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e00e854eb7fa621be8ca34bf: done: 0.018405 sec","job_status":"done","queue_duration_s":0.776576,"scheduling_latency_s":0.776598,"redis_calls":13,"redis_duration_s":0.00528,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002211,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003069,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010595,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018405,"completed_at":"2026-06-14T15:01:25.997Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1466vm-test-run-nixbot-gitlab> gitlab # [  147.997886] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:25.999Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"972ce167424de817b8a6893a","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-972ce167424de817b8a6893a: start","job_status":"start","queue_duration_s":0.7965,"scheduling_latency_s":0.796525}1467vm-test-run-nixbot-gitlab> gitlab # [  148.082182] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.015Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"972ce167424de817b8a6893a","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-972ce167424de817b8a6893a: done: 0.016109 sec","job_status":"done","queue_duration_s":0.7965,"scheduling_latency_s":0.796525,"redis_calls":13,"redis_duration_s":0.003028,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000917,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002111,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009901,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016109,"completed_at":"2026-06-14T15:01:26.015Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1468vm-test-run-nixbot-gitlab> gitlab # [  148.086315] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.016Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"451dd223831b67b929159dbb","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-451dd223831b67b929159dbb: start","job_status":"start","queue_duration_s":0.814177,"scheduling_latency_s":0.814198}1469vm-test-run-nixbot-gitlab> gitlab # [  148.108126] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.031Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"451dd223831b67b929159dbb","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-451dd223831b67b929159dbb: done: 0.015221 sec","job_status":"done","queue_duration_s":0.814177,"scheduling_latency_s":0.814198,"redis_calls":13,"redis_duration_s":0.002969,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000963,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002006,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009872,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015221,"completed_at":"2026-06-14T15:01:26.031Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1470vm-test-run-nixbot-gitlab> gitlab # [  148.185338] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.033Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e3476753c9c8eec42d29ecae","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e3476753c9c8eec42d29ecae: start","job_status":"start","queue_duration_s":0.830838,"scheduling_latency_s":0.83086}1471vm-test-run-nixbot-gitlab> gitlab # [  148.196929] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.048Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e3476753c9c8eec42d29ecae","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e3476753c9c8eec42d29ecae: done: 0.015005 sec","job_status":"done","queue_duration_s":0.830838,"scheduling_latency_s":0.83086,"redis_calls":13,"redis_duration_s":0.002883,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00085,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002033,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010204,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015005,"completed_at":"2026-06-14T15:01:26.048Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1472vm-test-run-nixbot-gitlab> gitlab # [  148.285536] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.049Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b741604515ddf3b670d6da6","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b741604515ddf3b670d6da6: start","job_status":"start","queue_duration_s":0.847257,"scheduling_latency_s":0.847278}1473vm-test-run-nixbot-gitlab> gitlab # [  148.287084] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.064Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b741604515ddf3b670d6da6","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b741604515ddf3b670d6da6: done: 0.014831 sec","job_status":"done","queue_duration_s":0.847257,"scheduling_latency_s":0.847278,"redis_calls":13,"redis_duration_s":0.002716,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000848,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001868,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010123,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014831,"completed_at":"2026-06-14T15:01:26.064Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1474vm-test-run-nixbot-gitlab> gitlab # [  148.377932] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.065Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62aad596f9b4ec364abcead3","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62aad596f9b4ec364abcead3: start","job_status":"start","queue_duration_s":0.863441,"scheduling_latency_s":0.863461}1475vm-test-run-nixbot-gitlab> gitlab # [  148.379171] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.080Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62aad596f9b4ec364abcead3","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62aad596f9b4ec364abcead3: done: 0.014941 sec","job_status":"done","queue_duration_s":0.863441,"scheduling_latency_s":0.863461,"redis_calls":13,"redis_duration_s":0.002898,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001015,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001883,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010094,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014941,"completed_at":"2026-06-14T15:01:26.080Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1476vm-test-run-nixbot-gitlab> gitlab # [  148.471282] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.082Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b5e89348f14807f49f95359a","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b5e89348f14807f49f95359a: start","job_status":"start","queue_duration_s":0.879607,"scheduling_latency_s":0.879628}1477vm-test-run-nixbot-gitlab> gitlab # [  148.472253] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.101Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b5e89348f14807f49f95359a","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b5e89348f14807f49f95359a: done: 0.019484 sec","job_status":"done","queue_duration_s":0.879607,"scheduling_latency_s":0.879628,"redis_calls":13,"redis_duration_s":0.007135000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004649,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002486,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010364,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019484,"completed_at":"2026-06-14T15:01:26.101Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1478vm-test-run-nixbot-gitlab> gitlab # [  148.551543] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.104Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"36208ab846b0dacebd640f26","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-36208ab846b0dacebd640f26: start","job_status":"start","queue_duration_s":0.90226,"scheduling_latency_s":0.902281}1479vm-test-run-nixbot-gitlab> gitlab # [  148.552990] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.120Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"36208ab846b0dacebd640f26","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-36208ab846b0dacebd640f26: done: 0.01616 sec","job_status":"done","queue_duration_s":0.90226,"scheduling_latency_s":0.902281,"redis_calls":13,"redis_duration_s":0.003182,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001008,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002174,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009894,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01616,"completed_at":"2026-06-14T15:01:26.120Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1480vm-test-run-nixbot-gitlab> gitlab # [  148.653077] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.122Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"18032178b4bd58834934d3ea","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-18032178b4bd58834934d3ea: start","job_status":"start","queue_duration_s":0.919683,"scheduling_latency_s":0.919706}1481vm-test-run-nixbot-gitlab> gitlab # [  148.654433] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.137Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"18032178b4bd58834934d3ea","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-18032178b4bd58834934d3ea: done: 0.015677 sec","job_status":"done","queue_duration_s":0.919683,"scheduling_latency_s":0.919706,"redis_calls":13,"redis_duration_s":0.0033079999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001035,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002273,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009657,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015677,"completed_at":"2026-06-14T15:01:26.137Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1482vm-test-run-nixbot-gitlab> gitlab # [  148.726179] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.139Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e62a34c0a4389d14fc7bb3cb","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e62a34c0a4389d14fc7bb3cb: start","job_status":"start","queue_duration_s":0.936652,"scheduling_latency_s":0.936672}1483vm-test-run-nixbot-gitlab> gitlab # [  148.727881] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.154Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e62a34c0a4389d14fc7bb3cb","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e62a34c0a4389d14fc7bb3cb: done: 0.015361 sec","job_status":"done","queue_duration_s":0.936652,"scheduling_latency_s":0.936672,"redis_calls":13,"redis_duration_s":0.0029389999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000869,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00207,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009739,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015361,"completed_at":"2026-06-14T15:01:26.154Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1484vm-test-run-nixbot-gitlab> gitlab # [  148.800152] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.155Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b336f921a5a54a96e7d0d512","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b336f921a5a54a96e7d0d512: start","job_status":"start","queue_duration_s":0.953287,"scheduling_latency_s":0.953307}1485vm-test-run-nixbot-gitlab> gitlab # [  148.801274] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.170Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b336f921a5a54a96e7d0d512","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b336f921a5a54a96e7d0d512: done: 0.015039 sec","job_status":"done","queue_duration_s":0.953287,"scheduling_latency_s":0.953307,"redis_calls":13,"redis_duration_s":0.00298,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000921,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002059,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009646,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015039,"completed_at":"2026-06-14T15:01:26.170Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1486vm-test-run-nixbot-gitlab> gitlab # [  148.897678] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.172Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"87b23bc20dcda8aa98db02ee","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-87b23bc20dcda8aa98db02ee: start","job_status":"start","queue_duration_s":0.969784,"scheduling_latency_s":0.969805}1487vm-test-run-nixbot-gitlab> gitlab # [  148.899333] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.190Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"87b23bc20dcda8aa98db02ee","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-87b23bc20dcda8aa98db02ee: done: 0.018589 sec","job_status":"done","queue_duration_s":0.969784,"scheduling_latency_s":0.969805,"redis_calls":13,"redis_duration_s":0.004764,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002735,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002029,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010974,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018589,"completed_at":"2026-06-14T15:01:26.190Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1488vm-test-run-nixbot-gitlab> gitlab # [  148.983884] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.192Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a8ebbf42f4f82bbcf0c5cb43","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a8ebbf42f4f82bbcf0c5cb43: start","job_status":"start","queue_duration_s":0.990239,"scheduling_latency_s":0.990267}1489vm-test-run-nixbot-gitlab> gitlab # [  148.985368] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.226Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a8ebbf42f4f82bbcf0c5cb43","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a8ebbf42f4f82bbcf0c5cb43: done: 0.034032 sec","job_status":"done","queue_duration_s":0.990239,"scheduling_latency_s":0.990267,"redis_calls":13,"redis_duration_s":0.020238,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000901,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.019337,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010704,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.034032,"completed_at":"2026-06-14T15:01:26.226Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1490vm-test-run-nixbot-gitlab> gitlab # [  149.060371] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.228Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f12cfda3dec6a227f5d0592","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f12cfda3dec6a227f5d0592: start","job_status":"start","queue_duration_s":1.025705,"scheduling_latency_s":1.025728}1491vm-test-run-nixbot-gitlab> gitlab # [  149.061861] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.243Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f12cfda3dec6a227f5d0592","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f12cfda3dec6a227f5d0592: done: 0.015249 sec","job_status":"done","queue_duration_s":1.025705,"scheduling_latency_s":1.025728,"redis_calls":13,"redis_duration_s":0.003057,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00105,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002007,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009898,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015249,"completed_at":"2026-06-14T15:01:26.243Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1492vm-test-run-nixbot-gitlab> gitlab # [  149.139325] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.244Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c10d63b92abac297327aedd1","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c10d63b92abac297327aedd1: start","job_status":"start","queue_duration_s":1.042404,"scheduling_latency_s":1.042424}1493vm-test-run-nixbot-gitlab> gitlab # [  149.139878] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.259Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c10d63b92abac297327aedd1","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c10d63b92abac297327aedd1: done: 0.015078 sec","job_status":"done","queue_duration_s":1.042404,"scheduling_latency_s":1.042424,"redis_calls":13,"redis_duration_s":0.0029569999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000887,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00207,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009765,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015078,"completed_at":"2026-06-14T15:01:26.259Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1494vm-test-run-nixbot-gitlab> gitlab # [  149.224445] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.261Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d67e16533c1722d56a9c9eb9","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d67e16533c1722d56a9c9eb9: start","job_status":"start","queue_duration_s":1.059249,"scheduling_latency_s":1.059271}1495vm-test-run-nixbot-gitlab> gitlab # [  149.225946] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.276Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d67e16533c1722d56a9c9eb9","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d67e16533c1722d56a9c9eb9: done: 0.01523 sec","job_status":"done","queue_duration_s":1.059249,"scheduling_latency_s":1.059271,"redis_calls":13,"redis_duration_s":0.002932,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000902,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00203,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010078,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01523,"completed_at":"2026-06-14T15:01:26.276Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1496vm-test-run-nixbot-gitlab> gitlab # [  149.306479] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.278Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e437dd7ce6a5e596c9af9c0f","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e437dd7ce6a5e596c9af9c0f: start","job_status":"start","queue_duration_s":1.075982,"scheduling_latency_s":1.076004}1497vm-test-run-nixbot-gitlab> gitlab # [  149.307381] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.296Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e437dd7ce6a5e596c9af9c0f","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e437dd7ce6a5e596c9af9c0f: done: 0.017886 sec","job_status":"done","queue_duration_s":1.075982,"scheduling_latency_s":1.076004,"redis_calls":13,"redis_duration_s":0.0032259999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001096,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00213,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012134,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017886,"completed_at":"2026-06-14T15:01:26.296Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1498vm-test-run-nixbot-gitlab> gitlab # [  149.393175] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.297Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"397d2095797cef971b7761da","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-397d2095797cef971b7761da: start","job_status":"start","queue_duration_s":1.095329,"scheduling_latency_s":1.095356}1499vm-test-run-nixbot-gitlab> gitlab # [  149.394797] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.314Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"397d2095797cef971b7761da","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-397d2095797cef971b7761da: done: 0.016517 sec","job_status":"done","queue_duration_s":1.095329,"scheduling_latency_s":1.095356,"redis_calls":13,"redis_duration_s":0.003077,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00091,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002167,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010695,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016517,"completed_at":"2026-06-14T15:01:26.314Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1500vm-test-run-nixbot-gitlab> gitlab # [  149.473266] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.315Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aa98b4346c4c294f1d48c0f8","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aa98b4346c4c294f1d48c0f8: start","job_status":"start","queue_duration_s":1.11323,"scheduling_latency_s":1.113253}1501vm-test-run-nixbot-gitlab> gitlab # [  149.474406] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.331Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aa98b4346c4c294f1d48c0f8","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aa98b4346c4c294f1d48c0f8: done: 0.015435 sec","job_status":"done","queue_duration_s":1.11323,"scheduling_latency_s":1.113253,"redis_calls":13,"redis_duration_s":0.002784,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000883,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001901,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010101,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015435,"completed_at":"2026-06-14T15:01:26.331Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1502vm-test-run-nixbot-gitlab> gitlab # [  149.546114] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.332Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"191b3c9064dc81230379b091","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-191b3c9064dc81230379b091: start","job_status":"start","queue_duration_s":1.129896,"scheduling_latency_s":1.129917}1503vm-test-run-nixbot-gitlab> gitlab # [  149.550327] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.347Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"191b3c9064dc81230379b091","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-191b3c9064dc81230379b091: done: 0.015308 sec","job_status":"done","queue_duration_s":1.129896,"scheduling_latency_s":1.129917,"redis_calls":13,"redis_duration_s":0.002969,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000868,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002101,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009585,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015308,"completed_at":"2026-06-14T15:01:26.347Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1504vm-test-run-nixbot-gitlab> gitlab # [  149.613233] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.349Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb06897fb88a92f57e2ca69a","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb06897fb88a92f57e2ca69a: start","job_status":"start","queue_duration_s":1.146496,"scheduling_latency_s":1.146517}1505vm-test-run-nixbot-gitlab> gitlab # [  149.692116] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.364Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb06897fb88a92f57e2ca69a","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb06897fb88a92f57e2ca69a: done: 0.01533 sec","job_status":"done","queue_duration_s":1.146496,"scheduling_latency_s":1.146517,"redis_calls":13,"redis_duration_s":0.0029389999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001038,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001901,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009691,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01533,"completed_at":"2026-06-14T15:01:26.364Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1506vm-test-run-nixbot-gitlab> gitlab # [  149.694151] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.365Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0185b2790e7a79cdde5d7313","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0185b2790e7a79cdde5d7313: start","job_status":"start","queue_duration_s":1.163126,"scheduling_latency_s":1.163147}1507vm-test-run-nixbot-gitlab> gitlab # [  149.714280] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.380Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0185b2790e7a79cdde5d7313","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0185b2790e7a79cdde5d7313: done: 0.015289 sec","job_status":"done","queue_duration_s":1.163126,"scheduling_latency_s":1.163147,"redis_calls":13,"redis_duration_s":0.00294,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000914,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002026,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009503,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015289,"completed_at":"2026-06-14T15:01:26.380Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1508vm-test-run-nixbot-gitlab> gitlab # [  149.813880] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.382Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e0863b142a606d7e5e9e8888","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e0863b142a606d7e5e9e8888: start","job_status":"start","queue_duration_s":1.179811,"scheduling_latency_s":1.179834}1509vm-test-run-nixbot-gitlab> gitlab # [  149.815311] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.397Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e0863b142a606d7e5e9e8888","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e0863b142a606d7e5e9e8888: done: 0.015251 sec","job_status":"done","queue_duration_s":1.179811,"scheduling_latency_s":1.179834,"redis_calls":13,"redis_duration_s":0.0029909999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001001,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00199,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009707,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015251,"completed_at":"2026-06-14T15:01:26.397Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1510vm-test-run-nixbot-gitlab> gitlab # [  149.858126] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.399Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e27bf20d94d2fcd7b659ca74","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e27bf20d94d2fcd7b659ca74: start","job_status":"start","queue_duration_s":1.196467,"scheduling_latency_s":1.196487}1511vm-test-run-nixbot-gitlab> gitlab # [  149.878169] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.414Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e27bf20d94d2fcd7b659ca74","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e27bf20d94d2fcd7b659ca74: done: 0.015128 sec","job_status":"done","queue_duration_s":1.196467,"scheduling_latency_s":1.196487,"redis_calls":13,"redis_duration_s":0.0030470000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000945,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002102,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009778,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015128,"completed_at":"2026-06-14T15:01:26.414Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1512vm-test-run-nixbot-gitlab> gitlab # [  149.941421] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.415Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"165e10b37fbeeb763a7e993e","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-165e10b37fbeeb763a7e993e: start","job_status":"start","queue_duration_s":1.21304,"scheduling_latency_s":1.213061}1513vm-test-run-nixbot-gitlab> gitlab # [  150.011986] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.431Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"165e10b37fbeeb763a7e993e","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-165e10b37fbeeb763a7e993e: done: 0.01566 sec","job_status":"done","queue_duration_s":1.21304,"scheduling_latency_s":1.213061,"redis_calls":13,"redis_duration_s":0.003117,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000945,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002172,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s"[  150.341199] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 3807815648 wd_nsec: 38078152861514vm-test-run-nixbot-gitlab> gitlab # :0.00997,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01566,"completed_at":"2026-06-14T15:01:26.431Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1515vm-test-run-nixbot-gitlab> gitlab # [  150.087630] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.432Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6a21b30c31d945a3b6b718d1","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6a21b30c31d945a3b6b718d1: start","job_status":"start","queue_duration_s":1.230256,"scheduling_latency_s":1.230277}1516vm-test-run-nixbot-gitlab> gitlab # [  150.090329] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.448Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6a21b30c31d945a3b6b718d1","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6a21b30c31d945a3b6b718d1: done: 0.01534 sec","job_status":"done","queue_duration_s":1.230256,"scheduling_latency_s":1.230277,"redis_calls":13,"redis_duration_s":0.0031349999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000921,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002214,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010004,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01534,"completed_at":"2026-06-14T15:01:26.448Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1517vm-test-run-nixbot-gitlab> gitlab # [  150.106962] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:26.449Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33c1e1a4f788e69c93fee400","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33c1e1a4f788e69c93fee400: start","job_status":"start","queue_duration_s":1.247118,"scheduling_latency_s":1.247139}1518vm-test-run-nixbot-gitlab> gitlab # [  150.221622] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.479Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33c1e1a4f788e69c93fee400","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33c1e1a4f788e69c93fee400: done: 3.030304 sec","job_status":"done","queue_duration_s":1.247118,"scheduling_latency_s":1.247139,"redis_calls":13,"redis_duration_s":0.003293,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00145,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001843,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015908,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":3.030304,"completed_at":"2026-06-14T15:01:29.479Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1519vm-test-run-nixbot-gitlab> gitlab # [  150.225391] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.487Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"541514af25ef85d5221eee11","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-541514af25ef85d5221eee11: start","job_status":"start","queue_duration_s":4.28483,"scheduling_latency_s":4.284877}1520vm-test-run-nixbot-gitlab> gitlab # [  150.298048] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.511Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"541514af25ef85d5221eee11","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-541514af25ef85d5221eee11: done: 0.024482 sec","job_status":"done","queue_duration_s":4.28483,"scheduling_latency_s":4.284877,"redis_calls":13,"redis_duration_s":0.003376,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001281,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002095,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015659,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024482,"completed_at":"2026-06-14T15:01:29.511Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1521vm-test-run-nixbot-gitlab> gitlab # [  150.300070] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.513Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e66ad87a45628964266e798f","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e66ad87a45628964266e798f: start","job_status":"start","queue_duration_s":4.31085,"scheduling_latency_s":4.310878}1522vm-test-run-nixbot-gitlab> gitlab # [  150.377887] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.529Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e66ad87a45628964266e798f","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e66ad87a45628964266e798f: done: 0.015715 sec","job_status":"done","queue_duration_s":4.31085,"scheduling_latency_s":4.310878,"redis_calls":13,"redis_duration_s":0.002694,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00082,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001874,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01035,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015715,"completed_at":"2026-06-14T15:01:29.529Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1523vm-test-run-nixbot-gitlab> gitlab # [  150.380074] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.530Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"520437eb3959fcff252e8368","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-520437eb3959fcff252e8368: start","job_status":"start","queue_duration_s":4.327767,"scheduling_latency_s":4.327789}1524vm-test-run-nixbot-gitlab> gitlab # [  150.461407] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.545Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"520437eb3959fcff252e8368","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-520437eb3959fcff252e8368: done: 0.014801 sec","job_status":"done","queue_duration_s":4.327767,"scheduling_latency_s":4.327789,"redis_calls":13,"redis_duration_s":0.002963,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000906,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002057,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009762,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014801,"completed_at":"2026-06-14T15:01:29.545Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1525vm-test-run-nixbot-gitlab> gitlab # [  150.463282] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.546Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c2232a49f5bb5d51846f18a","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c2232a49f5bb5d51846f18a: start","job_status":"start","queue_duration_s":4.343799,"scheduling_latency_s":4.34382}1526vm-test-run-nixbot-gitlab> gitlab # [  150.477139] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.560Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c2232a49f5bb5d51846f18a","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c2232a49f5bb5d51846f18a: done: 0.014662 sec","job_status":"done","queue_duration_s":4.343799,"scheduling_latency_s":4.34382,"redis_calls":13,"redis_duration_s":0.002709,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000843,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001866,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009786,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014662,"completed_at":"2026-06-14T15:01:29.560Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1527vm-test-run-nixbot-gitlab> gitlab # [  150.547303] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.562Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f1c0cb967e0e752042b690cb","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f1c0cb967e0e752042b690cb: start","job_status":"start","queue_duration_s":4.359762,"scheduling_latency_s":4.359784}1528vm-test-run-nixbot-gitlab> gitlab # [  150.623089] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.577Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f1c0cb967e0e752042b690cb","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f1c0cb967e0e752042b690cb: done: 0.014876 sec","job_status":"done","queue_duration_s":4.359762,"scheduling_latency_s":4.359784,"redis_calls":13,"redis_duration_s":0.002901,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00088,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002021,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009584,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014876,"completed_at":"2026-06-14T15:01:29.577Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1529vm-test-run-nixbot-gitlab> gitlab # [  150.625158] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.578Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e95a711ad2abd1e6ba9e4a9","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e95a711ad2abd1e6ba9e4a9: start","job_status":"start","queue_duration_s":4.375877,"scheduling_latency_s":4.375897}1530vm-test-run-nixbot-gitlab> gitlab # [  150.639259] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.592Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e95a711ad2abd1e6ba9e4a9","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e95a711ad2abd1e6ba9e4a9: done: 0.014592 sec","job_status":"done","queue_duration_s":4.375877,"scheduling_latency_s":4.375897,"redis_calls":13,"redis_duration_s":0.002716,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000821,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001895,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009823,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014592,"completed_at":"2026-06-14T15:01:29.592Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1531vm-test-run-nixbot-gitlab> gitlab # [  150.721150] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.594Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"96491e8fa2005869a794429b","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-96491e8fa2005869a794429b: start","job_status":"start","queue_duration_s":4.391727,"scheduling_latency_s":4.391747}1532vm-test-run-nixbot-gitlab> gitlab # [  150.722769] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.608Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"96491e8fa2005869a794429b","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-96491e8fa2005869a794429b: done: 0.014518 sec","job_status":"done","queue_duration_s":4.391727,"scheduling_latency_s":4.391747,"redis_calls":13,"redis_duration_s":0.002775,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000896,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001879,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009715,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014518,"completed_at":"2026-06-14T15:01:29.608Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1533vm-test-run-nixbot-gitlab> gitlab # [  150.820224] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.610Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9f257c17baada1df11415ace","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9f257c17baada1df11415ace: start","job_status":"start","queue_duration_s":4.407479,"scheduling_latency_s":4.4075}1534vm-test-run-nixbot-gitlab> gitlab # [  150.821355] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.624Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9f257c17baada1df11415ace","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9f257c17baada1df11415ace: done: 0.014729 sec","job_status":"done","queue_duration_s":4.407479,"scheduling_latency_s":4.4075,"redis_calls":13,"redis_duration_s":0.002807,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000869,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001938,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009768,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014729,"completed_at":"2026-06-14T15:01:29.624Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1535vm-test-run-nixbot-gitlab> gitlab # [  150.892297] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.625Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b1c224f564917de5589a353","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b1c224f564917de5589a353: start","job_status":"start","queue_duration_s":4.423425,"scheduling_latency_s":4.423446}1536vm-test-run-nixbot-gitlab> gitlab # [  150.894053] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.643Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b1c224f564917de5589a353","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b1c224f564917de5589a353: done: 0.01779 sec","job_status":"done","queue_duration_s":4.423425,"scheduling_latency_s":4.423446,"redis_calls":13,"redis_duration_s":0.003405,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001022,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002383,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011526,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01779,"completed_at":"2026-06-14T15:01:29.643Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1537vm-test-run-nixbot-gitlab> gitlab # [  151.007144] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.645Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc790f3768df66408eb98aeb","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc790f3768df66408eb98aeb: start","job_status":"start","queue_duration_s":4.442694,"scheduling_latency_s":4.442721}1538vm-test-run-nixbot-gitlab> gitlab # [  151.008291] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.660Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc790f3768df66408eb98aeb","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc790f3768df66408eb98aeb: done: 0.015422 sec","job_status":"done","queue_duration_s":4.442694,"scheduling_latency_s":4.442721,"redis_calls":13,"redis_duration_s":0.003013,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000917,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002096,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009638,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015422,"completed_at":"2026-06-14T15:01:29.660Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1539vm-test-run-nixbot-gitlab> gitlab # [  151.025370] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.661Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e0b8872d70b358c06b2ee2f","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e0b8872d70b358c06b2ee2f: start","job_status":"start","queue_duration_s":4.45935,"scheduling_latency_s":4.459371}1540vm-test-run-nixbot-gitlab> gitlab # [  151.045470] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.676Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e0b8872d70b358c06b2ee2f","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e0b8872d70b358c06b2ee2f: done: 0.014711 sec","job_status":"done","queue_duration_s":4.45935,"scheduling_latency_s":4.459371,"redis_calls":13,"redis_duration_s":0.002894,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000904,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00199,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009397,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014711,"completed_at":"2026-06-14T15:01:29.676Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1541vm-test-run-nixbot-gitlab> gitlab # [  151.114383] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.677Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4fdb5f28e3a23102a5b8077a","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4fdb5f28e3a23102a5b8077a: start","job_status":"start","queue_duration_s":4.475318,"scheduling_latency_s":4.475338}1542vm-test-run-nixbot-gitlab> gitlab # [  151.126295] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.692Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4fdb5f28e3a23102a5b8077a","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4fdb5f28e3a23102a5b8077a: done: 0.01462 sec","job_status":"done","queue_duration_s":4.475318,"scheduling_latency_s":4.475338,"redis_calls":13,"redis_duration_s":0.002885,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000936,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001949,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009369,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01462,"completed_at":"2026-06-14T15:01:29.692Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1543vm-test-run-nixbot-gitlab> gitlab # [  151.214261] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.693Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d26ebd6625fe41c181cf1c98","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d26ebd6625fe41c181cf1c98: start","job_status":"start","queue_duration_s":4.491231,"scheduling_latency_s":4.491253}1544vm-test-run-nixbot-gitlab> gitlab # [  151.215443] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.708Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d26ebd6625fe41c181cf1c98","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d26ebd6625fe41c181cf1c98: done: 0.014604 sec","job_status":"done","queue_duration_s":4.491231,"scheduling_latency_s":4.491253,"redis_calls":13,"redis_duration_s":0.002854,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000914,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00194,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009302,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014604,"completed_at":"2026-06-14T15:01:29.708Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1545vm-test-run-nixbot-gitlab> gitlab # [  151.294367] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.709Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e2498b8c54e569c44af3e11","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e2498b8c54e569c44af3e11: start","job_status":"start","queue_duration_s":4.50712,"scheduling_latency_s":4.507142}1546vm-test-run-nixbot-gitlab> gitlab # [  151.295205] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.724Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e2498b8c54e569c44af3e11","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e2498b8c54e569c44af3e11: done: 0.014635 sec","job_status":"done","queue_duration_s":4.50712,"scheduling_latency_s":4.507142,"redis_calls":13,"redis_duration_s":0.0030139999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000908,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002106,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009348,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014635,"completed_at":"2026-06-14T15:01:29.724Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1547vm-test-run-nixbot-gitlab> gitlab # [  151.414940] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.725Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b513e5eead2ea23eb870f544","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b513e5eead2ea23eb870f544: start","job_status":"start","queue_duration_s":4.52307,"scheduling_latency_s":4.523092}1548vm-test-run-nixbot-gitlab> gitlab # [  151.416548] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.740Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b513e5eead2ea23eb870f544","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b513e5eead2ea23eb870f544: done: 0.014614 sec","job_status":"done","queue_duration_s":4.52307,"scheduling_latency_s":4.523092,"redis_calls":13,"redis_duration_s":0.003001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000912,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002089,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009255,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014614,"completed_at":"2026-06-14T15:01:29.740Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1549vm-test-run-nixbot-gitlab> gitlab # [  151.487461] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.741Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ebea02ce7bc5087ac27925ab","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ebea02ce7bc5087ac27925ab: start","job_status":"start","queue_duration_s":4.53916,"scheduling_latency_s":4.539181}1550vm-test-run-nixbot-gitlab> gitlab # [  151.488888] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.756Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ebea02ce7bc5087ac27925ab","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ebea02ce7bc5087ac27925ab: done: 0.014578 sec","job_status":"done","queue_duration_s":4.53916,"scheduling_latency_s":4.539181,"redis_calls":13,"redis_duration_s":0.0029850000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000906,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002079,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009274,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014578,"completed_at":"2026-06-14T15:01:29.756Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1551vm-test-run-nixbot-gitlab> gitlab # [  151.513927] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.757Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b3377787eede8c37f534676","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b3377787eede8c37f534676: start","job_status":"start","queue_duration_s":4.555031,"scheduling_latency_s":4.555051}1552vm-test-run-nixbot-gitlab> gitlab # [  151.534346] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.772Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b3377787eede8c37f534676","created_at":"2026-06-14T15:01:25.135Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-0e135ba2a5784277","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"889460c0bf613512092b3a90cc7bfa82","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.202Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b3377787eede8c37f534676: done: 0.014671 sec","job_status":"done","queue_duration_s":4.555031,"scheduling_latency_s":4.555051,"redis_calls":13,"redis_duration_s":0.003025,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000915,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00211,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00931,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014671,"completed_at":"2026-06-14T15:01:29.772Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1553vm-test-run-nixbot-gitlab> gitlab # [  151.597273] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.773Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"85a7a50fe4cdc97eea2b35a9","created_at":"2026-06-14T15:01:25.619Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-ef48ec209ec2454b","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"b9b890b1f88389ded1e7e55e6de07fb5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.630Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-85a7a50fe4cdc97eea2b35a9: start","job_status":"start","queue_duration_s":4.143346,"scheduling_latency_s":4.143367}1554vm-test-run-nixbot-gitlab> gitlab # [  151.617261] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.787Z","message":"No existing merge request to be cleaned up."}1555vm-test-run-nixbot-gitlab> gitlab # [  151.620125] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.796Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"85a7a50fe4cdc97eea2b35a9","created_at":"2026-06-14T15:01:25.619Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-ef48ec209ec2454b","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"b9b890b1f88389ded1e7e55e6de07fb5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.630Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-85a7a50fe4cdc97eea2b35a9: done: 0.022641 sec","job_status":"done","queue_duration_s":4.143346,"scheduling_latency_s":4.143367,"redis_calls":13,"redis_duration_s":0.002941,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000458,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001128,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001355,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013845,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000356749,"duration_s":0.022641,"completed_at":"2026-06-14T15:01:29.796Z","load_balancing_strategy":"primary","db_duration_s":0.00313,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1556vm-test-run-nixbot-gitlab> gitlab # [  151.715568] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.797Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4eed8f003aa147a6909e3053","created_at":"2026-06-14T15:01:25.619Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-ef48ec209ec2454b","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"b9b890b1f88389ded1e7e55e6de07fb5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.630Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4eed8f003aa147a6909e3053: start","job_status":"start","queue_duration_s":4.167227,"scheduling_latency_s":4.167247}1557vm-test-run-nixbot-gitlab> gitlab # [  151.717245] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.806Z","message":"No existing merge request to be cleaned up."}1558vm-test-run-nixbot-gitlab> gitlab # [  151.718966] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.812Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4eed8f003aa147a6909e3053","created_at":"2026-06-14T15:01:25.619Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-ef48ec209ec2454b","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"b9b890b1f88389ded1e7e55e6de07fb5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.630Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4eed8f003aa147a6909e3053: done: 0.015349 sec","job_status":"done","queue_duration_s":4.167227,"scheduling_latency_s":4.167247,"redis_calls":10,"redis_duration_s":0.002345,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000877,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001468,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009501,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015349,"completed_at":"2026-06-14T15:01:29.812Z","load_balancing_strategy":"primary","db_duration_s":0.001768,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1559vm-test-run-nixbot-gitlab> gitlab # [  151.825346] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.814Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"eb50e522f5d1480eaeb2f43f","created_at":"2026-06-14T15:01:25.619Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-ef48ec209ec2454b","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"b9b890b1f88389ded1e7e55e6de07fb5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.630Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-eb50e522f5d1480eaeb2f43f: start","job_status":"start","queue_duration_s":4.183834,"scheduling_latency_s":4.183856}1560vm-test-run-nixbot-gitlab> gitlab # [  151.827189] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.823Z","message":"No existing merge request to be cleaned up."}1561vm-test-run-nixbot-gitlab> gitlab # [  151.828832] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.828Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"eb50e522f5d1480eaeb2f43f","created_at":"2026-06-14T15:01:25.619Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-ef48ec209ec2454b","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"b9b890b1f88389ded1e7e55e6de07fb5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.630Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-eb50e522f5d1480eaeb2f43f: done: 0.014995 sec","job_status":"done","queue_duration_s":4.183834,"scheduling_latency_s":4.183856,"redis_calls":10,"redis_duration_s":0.00235,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000905,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001445,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009378,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014995,"completed_at":"2026-06-14T15:01:29.828Z","load_balancing_strategy":"primary","db_duration_s":0.001584,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1562vm-test-run-nixbot-gitlab> gitlab # [  151.849264] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.830Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"62ea80d7903b9762f7feec03","created_at":"2026-06-14T15:01:25.619Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-ef48ec209ec2454b","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"b9b890b1f88389ded1e7e55e6de07fb5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.630Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-62ea80d7903b9762f7feec03: start","job_status":"start","queue_duration_s":4.200111,"scheduling_latency_s":4.200132}1563vm-test-run-nixbot-gitlab> gitlab # [  151.868777] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.839Z","message":"No existing merge request to be cleaned up."}1564vm-test-run-nixbot-gitlab> gitlab # [  151.871371] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:01:29.845Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"62ea80d7903b9762f7feec03","created_at":"2026-06-14T15:01:25.619Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-ef48ec209ec2454b","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"b9b890b1f88389ded1e7e55e6de07fb5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:01:25.630Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-62ea80d7903b9762f7feec03: done: 0.015485 sec","job_status":"done","queue_duration_s":4.200111,"scheduling_latency_s":4.200132,"redis_calls":10,"redis_duration_s":0.002301,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000962,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001339,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00962,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015485,"completed_at":"2026-06-14T15:01:29.845Z","load_balancing_strategy":"primary","db_duration_s":0.001502,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1565vm-test-run-nixbot-gitlab> gitlab # [  162.989784] setup-gitlab-start[2670]: W, [2026-06-14T15:01:42.251187 #2670]  WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1566vm-test-run-nixbot-gitlab> gitlab # [  182.346440] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:01.606Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"de517227a40e6f9fdfe81f4c","created_at":"2026-06-14T15:02:01.550Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dda4981a443a49c2994d4a02775e7d62","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:01.604Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-de517227a40e6f9fdfe81f4c: start","job_status":"start","queue_duration_s":0.002689,"scheduling_latency_s":0.002721}1567vm-test-run-nixbot-gitlab> gitlab # [  182.427782] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:01.689Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1568vm-test-run-nixbot-gitlab> gitlab # [  182.448505] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:01.710Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"de517227a40e6f9fdfe81f4c","created_at":"2026-06-14T15:02:01.550Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dda4981a443a49c2994d4a02775e7d62","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:01.604Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-de517227a40e6f9fdfe81f4c: done: 0.103183 sec","job_status":"done","queue_duration_s":0.002689,"scheduling_latency_s":0.002721,"redis_calls":7,"redis_duration_s":0.017605,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.015759,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001846,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018134,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.103183,"completed_at":"2026-06-14T15:02:01.709Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1569vm-test-run-nixbot-gitlab> gitlab # [  182.532813] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:01.794Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"921ef95429afbf8493d9b2db","created_at":"2026-06-14T15:02:01.684Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"64fc84ce869b3ac9fba219455da77856","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:01.712Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-921ef95429afbf8493d9b2db: start","job_status":"start","queue_duration_s":0.082037,"scheduling_latency_s":0.082229}1570vm-test-run-nixbot-gitlab> gitlab # [  182.632193] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:01.893Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"921ef95429afbf8493d9b2db","created_at":"2026-06-14T15:02:01.684Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"64fc84ce869b3ac9fba219455da77856","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:01.712Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-921ef95429afbf8493d9b2db: done: 0.099486 sec","job_status":"done","queue_duration_s":0.082037,"scheduling_latency_s":0.082229,"redis_calls":12,"redis_duration_s":0.017787,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.003112,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001668,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.011078,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001929,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028963,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.004211988,"duration_s":0.099486,"completed_at":"2026-06-14T15:02:01.893Z","load_balancing_strategy":"primary","db_duration_s":0.009848,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1571vm-test-run-nixbot-gitlab> gitlab # [  182.714863] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:01.976Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Gitlab::JiraImport::StuckJiraImportJobsWorker","jid":"caa3a239dec0b67b6b0d1e80","created_at":"2026-06-14T15:02:01.827Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"13c91154455b62a22bff0f2d612732ee","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b9cf6c9b6c2f8fad80e08693d59af787bb65ad6ffe24ec5d0e1a669517566127","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:01.835Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Gitlab::JiraImport::StuckJiraImportJobsWorker JID-caa3a239dec0b67b6b0d1e80: start","job_status":"start","queue_duration_s":0.140676,"scheduling_latency_s":0.140738}1572vm-test-run-nixbot-gitlab> gitlab # [  182.801827] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:02.063Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Gitlab::JiraImport::StuckJiraImportJobsWorker","jid":"caa3a239dec0b67b6b0d1e80","created_at":"2026-06-14T15:02:01.827Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"13c91154455b62a22bff0f2d612732ee","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b9cf6c9b6c2f8fad80e08693d59af787bb65ad6ffe24ec5d0e1a669517566127","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:01.835Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Gitlab::JiraImport::StuckJiraImportJobsWorker JID-caa3a239dec0b67b6b0d1e80: done: 0.086757 sec","job_status":"done","queue_duration_s":0.140676,"scheduling_latency_s":0.140738,"redis_calls":7,"redis_duration_s":0.021049,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.017432,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003617,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022324,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.086757,"completed_at":"2026-06-14T15:02:02.063Z","load_balancing_strategy":"primary","db_duration_s":0.010684,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1573vm-test-run-nixbot-gitlab> gitlab # [  182.889386] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:02.150Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"a3dd7f348d65b526ca7aade6","created_at":"2026-06-14T15:02:01.850Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7b6c6b4ce736b5f62d916a6b76f229cd","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:01.854Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-a3dd7f348d65b526ca7aade6: start","job_status":"start","queue_duration_s":0.296566,"scheduling_latency_s":0.296627}1574vm-test-run-nixbot-gitlab> gitlab # [  183.692357] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:02.953Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"a3dd7f348d65b526ca7aade6","created_at":"2026-06-14T15:02:01.850Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7b6c6b4ce736b5f62d916a6b76f229cd","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:01.854Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-a3dd7f348d65b526ca7aade6: done: 0.802655 sec","job_status":"done","queue_duration_s":0.296566,"scheduling_latency_s":0.296627,"redis_calls":571,"redis_duration_s":0.400011,"redis_read_bytes":569,"redis_write_bytes":49487,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003162,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.396849,"redis_shared_state_read_bytes":565,"redis_shared_state_write_bytes":49017,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.436912,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.802655,"completed_at":"2026-06-14T15:02:02.953Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1575vm-test-run-nixbot-gitlab> gitlab # [  183.759964] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:03.021Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"56d2dfa627b885010f9adf87","created_at":"2026-06-14T15:02:01.862Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0c4c86556ec953cd7489313c7b5a26aa","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:01.866Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-56d2dfa627b885010f9adf87: start","job_status":"start","queue_duration_s":1.155495,"scheduling_latency_s":1.155564}1576vm-test-run-nixbot-gitlab> gitlab # [  184.048938] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:03.310Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"56d2dfa627b885010f9adf87","created_at":"2026-06-14T15:02:01.862Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0c4c86556ec953cd7489313c7b5a26aa","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:01.866Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-56d2dfa627b885010f9adf87: done: 0.28944 sec","job_status":"done","queue_duration_s":1.155495,"scheduling_latency_s":1.155564,"redis_calls":11,"redis_duration_s":0.006022,"redis_read_bytes":9,"redis_write_bytes":1262,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001219,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":275,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002566,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002237,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":124,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":124,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.053,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":124,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":80,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.053,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.187221,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001228648,"exclusive_lock_hold_duration_s":0.235420423,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.28944,"completed_at":"2026-06-14T15:02:03.310Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.086512,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1577vm-test-run-nixbot-gitlab> gitlab # [  184.152747] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:03.414Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"849f5479536f12f9b70e7dea","created_at":"2026-06-14T15:02:01.879Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"66460af793250df43d424bf8496a1fd9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:01.885Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-849f5479536f12f9b70e7dea: start","job_status":"start","queue_duration_s":1.52955,"scheduling_latency_s":1.529587}1578vm-test-run-nixbot-gitlab> gitlab # [  184.205467] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:03.466Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"849f5479536f12f9b70e7dea","created_at":"2026-06-14T15:02:01.879Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"66460af793250df43d424bf8496a1fd9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:01.885Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-849f5479536f12f9b70e7dea: done: 0.052359 sec","job_status":"done","queue_duration_s":1.52955,"scheduling_latency_s":1.529587,"redis_calls":9,"redis_duration_s":0.005563,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002898,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002665,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021287,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001378667,"exclusive_lock_hold_duration_s":0.004445537,"duration_s":0.052359,"completed_at":"2026-06-14T15:02:03.466Z","load_balancing_strategy":"primary","db_duration_s":0.001086,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1579vm-test-run-nixbot-gitlab> gitlab # [  184.270898] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:03.532Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"41c1495a9975b6399681daa6","created_at":"2026-06-14T15:02:01.977Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1d937ef9ad83ac9092e0d7ff1210ecc6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:02.008Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-41c1495a9975b6399681daa6: start","job_status":"start","queue_duration_s":1.524655,"scheduling_latency_s":1.52471}1580vm-test-run-nixbot-gitlab> gitlab # [  184.313892] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:03.575Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1581vm-test-run-nixbot-gitlab> gitlab # [  184.324940] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:03.586Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"41c1495a9975b6399681daa6","created_at":"2026-06-14T15:02:01.977Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1d937ef9ad83ac9092e0d7ff1210ecc6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:02.008Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-41c1495a9975b6399681daa6: done: 0.053981 sec","job_status":"done","queue_duration_s":1.524655,"scheduling_latency_s":1.52471,"redis_calls":7,"redis_duration_s":0.0029460000000000003,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002089,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000857,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017722,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.053981,"completed_at":"2026-06-14T15:02:03.586Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1582vm-test-run-nixbot-gitlab> gitlab # [  184.327556] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:03.589Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"93e135979fc7824bf586116d","created_at":"2026-06-14T15:02:02.029Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:02.035Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-93e135979fc7824bf586116d: start","job_status":"start","queue_duration_s":1.553642,"scheduling_latency_s":1.553687}1583vm-test-run-nixbot-gitlab> gitlab # [  184.410841] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:03.672Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"93e135979fc7824bf586116d","created_at":"2026-06-14T15:02:02.029Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:02.035Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-93e135979fc7824bf586116d: done: 0.083334 sec","job_status":"done","queue_duration_s":1.553642,"scheduling_latency_s":1.553687,"redis_calls":60,"redis_duration_s":0.016863,"redis_read_bytes":108,"redis_write_bytes":41059,"redis_queues_calls":2,"redis_queues_duration_s":0.001172,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.014624,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001067,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.06487,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.083334,"completed_at":"2026-06-14T15:02:03.672Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1584vm-test-run-nixbot-gitlab> gitlab # [  184.477327] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:03.739Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"fb4da5f4a94bc0ee25b0622e","created_at":"2026-06-14T15:02:02.056Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9bb31adc51c1d49d3a78ed99f3b01b17","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:02.060Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-fb4da5f4a94bc0ee25b0622e: start","job_status":"start","queue_duration_s":1.679052,"scheduling_latency_s":1.679091}1585vm-test-run-nixbot-gitlab> gitlab # [  184.488155] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:03.750Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1586vm-test-run-nixbot-gitlab> gitlab # [  184.497468] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:03.759Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"fb4da5f4a94bc0ee25b0622e","created_at":"2026-06-14T15:02:02.056Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9bb31adc51c1d49d3a78ed99f3b01b17","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:02.060Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-fb4da5f4a94bc0ee25b0622e: done: 0.01998 sec","job_status":"done","queue_duration_s":1.679052,"scheduling_latency_s":1.679091,"redis_calls":7,"redis_duration_s":0.003548,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002625,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000923,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011308,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01998,"completed_at":"2026-06-14T15:02:03.759Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1587vm-test-run-nixbot-gitlab> gitlab # [  184.499714] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:03.761Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"f1d1e156bb9e1c384690e2cf","created_at":"2026-06-14T15:02:02.152Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1b685063099da41185a9273a98828548","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:02.183Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-f1d1e156bb9e1c384690e2cf: start","job_status":"start","queue_duration_s":1.578493,"scheduling_latency_s":1.578533}1588vm-test-run-nixbot-gitlab> gitlab # [  184.616613] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:03.878Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"f1d1e156bb9e1c384690e2cf","created_at":"2026-06-14T15:02:02.152Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1b685063099da41185a9273a98828548","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:02.183Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-f1d1e156bb9e1c384690e2cf: done: 0.116228 sec","job_status":"done","queue_duration_s":1.578493,"scheduling_latency_s":1.578533,"redis_calls":7,"redis_duration_s":0.002971,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00218,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000791,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018855,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.116228,"completed_at":"2026-06-14T15:02:03.877Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.090445,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1589vm-test-run-nixbot-gitlab> gitlab # [  184.653725] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:03.881Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"c9abd6088cf6dd892f7f451f","created_at":"2026-06-14T15:02:02.201Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"415cbeba454e34012ff925793a2ea58e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-14T15:02:02.207Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-c9abd6088cf6dd892f7f451f: start","job_status":"start","queue_duration_s":1.673652,"scheduling_latency_s":1.673708}1590vm-test-run-nixbot-gitlab> gitlab # [  184.655471] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:03.900Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1591vm-test-run-nixbot-gitlab> gitlab # [  184.656879] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:03.911Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"c9abd6088cf6dd892f7f451f","created_at":"2026-06-14T15:02:02.201Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"415cbeba454e34012ff925793a2ea58e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-14T15:02:02.207Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-c9abd6088cf6dd892f7f451f: done: 0.030194 sec","job_status":"done","queue_duration_s":1.673652,"scheduling_latency_s":1.673708,"redis_calls":7,"redis_duration_s":0.004455,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003173,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001282,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01793,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030194,"completed_at":"2026-06-14T15:02:03.911Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1592vm-test-run-nixbot-gitlab> gitlab # [  184.761471] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:03.914Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"0a734827707aada9359bd756","created_at":"2026-06-14T15:02:02.224Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a4b6bf9307d75de6c6947bd6384db66a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-14T15:02:02.229Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-0a734827707aada9359bd756: start","job_status":"start","queue_duration_s":1.684944,"scheduling_latency_s":1.684984}1593vm-test-run-nixbot-gitlab> gitlab # [  184.834217] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:04.095Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"0a734827707aada9359bd756","created_at":"2026-06-14T15:02:02.224Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a4b6bf9307d75de6c6947bd6384db66a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-14T15:02:02.229Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-0a734827707aada9359bd756: done: 0.181335 sec","job_status":"done","queue_duration_s":1.684944,"scheduling_latency_s":1.684984,"redis_calls":8,"redis_duration_s":0.003248,"redis_read_bytes":133,"redis_write_bytes":811,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000387,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":70,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002045,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000816,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029852,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.181335,"completed_at":"2026-06-14T15:02:04.095Z","load_balancing_strategy":"primary","db_duration_s":0.001795,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1594vm-test-run-nixbot-gitlab> gitlab # [  184.902612] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:04.164Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"dd1c3918f94aeea250bb8401","created_at":"2026-06-14T15:02:02.258Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5b88384f211951a135c07e5d55a3e316","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:02.264Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-dd1c3918f94aeea250bb8401: start","job_status":"start","queue_duration_s":1.899296,"scheduling_latency_s":1.899379}1595vm-test-run-nixbot-gitlab> gitlab # [  184.936727] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:04.198Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"dd1c3918f94aeea250bb8401","created_at":"2026-06-14T15:02:02.258Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5b88384f211951a135c07e5d55a3e316","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:02.264Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-dd1c3918f94aeea250bb8401: done: 0.034333 sec","job_status":"done","queue_duration_s":1.899296,"scheduling_latency_s":1.899379,"redis_calls":7,"redis_duration_s":0.004252,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003172,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00108,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018187,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034333,"completed_at":"2026-06-14T15:02:04.198Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003809,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1596vm-test-run-nixbot-gitlab> gitlab # [  185.005236] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:04.266Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"317ad39b2f2dbaa17cfb5057","created_at":"2026-06-14T15:02:02.301Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f3b21432cb86e754d64b3c25bf76cba0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:02.306Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-317ad39b2f2dbaa17cfb5057: start","job_status":"start","queue_duration_s":1.96064,"scheduling_latency_s":1.96071}1597vm-test-run-nixbot-gitlab> gitlab # [  185.057847] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:04.319Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"317ad39b2f2dbaa17cfb5057","created_at":"2026-06-14T15:02:02.301Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f3b21432cb86e754d64b3c25bf76cba0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:02.306Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-317ad39b2f2dbaa17cfb5057: done: 0.052594 sec","job_status":"done","queue_duration_s":1.96064,"scheduling_latency_s":1.96071,"redis_calls":14,"redis_duration_s":0.008467,"redis_read_bytes":16,"redis_write_bytes":4022,"redis_queues_calls":2,"redis_queues_duration_s":0.000812,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3011,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.005708,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001947,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034067,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.052594,"completed_at":"2026-06-14T15:02:04.319Z","load_balancing_strategy":"primary","db_duration_s":0.001996,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1598vm-test-run-nixbot-gitlab> gitlab # [  185.126234] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:04.387Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","jid":"5c4b48ac6dc0f4b9932af27a","created_at":"2026-06-14T15:02:02.318Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0e5940dc85f904097da1997d80fd3735","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:805014656d48c4962f83ad00758ac8cfab4de244cfbd1d771454e66bd9018911","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:02.329Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker JID-5c4b48ac6dc0f4b9932af27a: start","job_status":"start","queue_duration_s":2.058062,"scheduling_latency_s":2.058125}1599vm-test-run-nixbot-gitlab> gitlab # [  185.200816] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:04.462Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","jid":"5c4b48ac6dc0f4b9932af27a","created_at":"2026-06-14T15:02:02.318Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0e5940dc85f904097da1997d80fd3735","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:805014656d48c4962f83ad00758ac8cfab4de244cfbd1d771454e66bd9018911","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:02.329Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker JID-5c4b48ac6dc0f4b9932af27a: done: 0.074545 sec","job_status":"done","queue_duration_s":2.058062,"scheduling_latency_s":2.058125,"redis_calls":20,"redis_duration_s":0.011568,"redis_read_bytes":20,"redis_write_bytes":3247,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.004694,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1346,"redis_queues_calls":2,"redis_queues_duration_s":0.000806,"redis_queues_read_bytes":3,"redis_queues_write_bytes":817,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.004114,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":734,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001954,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":350,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.048184,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000916038,"duration_s":0.074545,"completed_at":"2026-06-14T15:02:04.462Z","load_balancing_strategy":"primary","db_duration_s":0.005994,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1600vm-test-run-nixbot-gitlab> gitlab # [  185.281126] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:04.542Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"cebc97dc3f66c8e6cccaa7a8","created_at":"2026-06-14T15:02:02.340Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e3e2bd9935c7ac8a51e3499670664c7e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:02.343Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-cebc97dc3f66c8e6cccaa7a8: start","job_status":"start","queue_duration_s":2.1987,"scheduling_latency_s":2.198764}1601vm-test-run-nixbot-gitlab> gitlab # [  185.313406] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:04.575Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"cebc97dc3f66c8e6cccaa7a8","created_at":"2026-06-14T15:02:02.340Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e3e2bd9935c7ac8a51e3499670664c7e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:02.343Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-cebc97dc3f66c8e6cccaa7a8: done: 0.03249 sec","job_status":"done","queue_duration_s":2.1987,"scheduling_latency_s":2.198764,"redis_calls":7,"redis_duration_s":0.004131,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003012,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001119,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019361,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.03249,"completed_at":"2026-06-14T15:02:04.574Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1602vm-test-run-nixbot-gitlab> gitlab # [  185.349801] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:04.578Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::CreateStatisticsWorker","jid":"ec31c06f294fa7f2b2e782d1","created_at":"2026-06-14T15:02:02.367Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"21ca5d0438f00473cd5f019bf453be51","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5d77e61ff59f270527c3c7951a3b25efb5fc91aa5024f64a2a8e994fa06363fd","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:02.379Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::CreateStatisticsWorker JID-ec31c06f294fa7f2b2e782d1: start","job_status":"start","queue_duration_s":2.19913,"scheduling_latency_s":2.199179}1603vm-test-run-nixbot-gitlab> gitlab # [  185.539606] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:04.801Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::CreateStatisticsWorker","jid":"ec31c06f294fa7f2b2e782d1","created_at":"2026-06-14T15:02:02.367Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"21ca5d0438f00473cd5f019bf453be51","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5d77e61ff59f270527c3c7951a3b25efb5fc91aa5024f64a2a8e994fa06363fd","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:02.379Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::CreateStatisticsWorker JID-ec31c06f294fa7f2b2e782d1: done: 0.222734 sec","job_status":"done","queue_duration_s":2.19913,"scheduling_latency_s":2.199179,"redis_calls":10,"redis_duration_s":0.005087,"redis_read_bytes":9,"redis_write_bytes":1047,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000664,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003232,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001191,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":29,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":29,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":29,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.078835,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000900673,"duration_s":0.222734,"completed_at":"2026-06-14T15:02:04.800Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.041216,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1604vm-test-run-nixbot-gitlab> gitlab # [  185.651447] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:04.913Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"5ef0bd38f36e0c7f21b95936","created_at":"2026-06-14T15:02:02.393Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fa3577c256dd94375e498177c1000396","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:02.398Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-5ef0bd38f36e0c7f21b95936: start","job_status":"start","queue_duration_s":2.513947,"scheduling_latency_s":2.514011}1605vm-test-run-nixbot-gitlab> gitlab # [  185.703611] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:04.965Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"5ef0bd38f36e0c7f21b95936","created_at":"2026-06-14T15:02:02.393Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fa3577c256dd94375e498177c1000396","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:02.398Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-5ef0bd38f36e0c7f21b95936: done: 0.052188 sec","job_status":"done","queue_duration_s":2.513947,"scheduling_latency_s":2.514011,"redis_calls":7,"redis_duration_s":0.002954,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002065,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000889,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01951,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.052188,"completed_at":"2026-06-14T15:02:04.964Z","load_balancing_strategy":"primary","db_duration_s":0.002376,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1606vm-test-run-nixbot-gitlab> gitlab # [  185.771890] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:05.033Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"b0a9a06407ae9408fe59afe7","created_at":"2026-06-14T15:02:02.419Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bb3000346b219dfb6512d7301f857536","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:02.425Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-b0a9a06407ae9408fe59afe7: start","job_status":"start","queue_duration_s":2.60763,"scheduling_latency_s":2.607695}1607vm-test-run-nixbot-gitlab> gitlab # [  185.789824] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:05.051Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1608vm-test-run-nixbot-gitlab> gitlab # [  185.803272] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:05.064Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"b0a9a06407ae9408fe59afe7","created_at":"2026-06-14T15:02:02.419Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bb3000346b219dfb6512d7301f857536","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:02.425Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-b0a9a06407ae9408fe59afe7: done: 0.031963 sec","job_status":"done","queue_duration_s":2.60763,"scheduling_latency_s":2.607695,"redis_calls":7,"redis_duration_s":0.004041,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002792,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001249,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018746,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031963,"completed_at":"2026-06-14T15:02:05.064Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1609vm-test-run-nixbot-gitlab> gitlab # [  185.871500] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:05.133Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"f7d26177772b7c5cc77b63ee","created_at":"2026-06-14T15:02:02.443Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c0482c19d8833f5ac44dfe840e8c094e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:02.450Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-f7d26177772b7c5cc77b63ee: start","job_status":"start","queue_duration_s":2.682593,"scheduling_latency_s":2.682674}1610vm-test-run-nixbot-gitlab> gitlab # [  185.904451] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:05.165Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"f7d26177772b7c5cc77b63ee","created_at":"2026-06-14T15:02:02.443Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c0482c19d8833f5ac44dfe840e8c094e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:02.450Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-f7d26177772b7c5cc77b63ee: done: 0.033004 sec","job_status":"done","queue_duration_s":2.682593,"scheduling_latency_s":2.682674,"redis_calls":7,"redis_duration_s":0.004242,"redis_read_bytes":5,"redis_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002967,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001275,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019521,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033004,"completed_at":"2026-06-14T15:02:05.165Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1611vm-test-run-nixbot-gitlab> gitlab # [  185.972263] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:05.233Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"e56bd63615b3ccadcf681b22","created_at":"2026-06-14T15:02:02.468Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"96792904e4283ec66965e186d0be6f5c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:02.475Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-e56bd63615b3ccadcf681b22: start","job_status":"start","queue_duration_s":2.758249,"scheduling_latency_s":2.758313}1612vm-test-run-nixbot-gitlab> gitlab # [  186.033450] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:05.294Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"e56bd63615b3ccadcf681b22","created_at":"2026-06-14T15:02:02.468Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"96792904e4283ec66965e186d0be6f5c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:02.475Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-e56bd63615b3ccadcf681b22: done: 0.061091 sec","job_status":"done","queue_duration_s":2.758249,"scheduling_latency_s":2.758313,"redis_calls":9,"redis_duration_s":0.005552,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002983,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002569,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033541,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001242895,"exclusive_lock_hold_duration_s":0.028868473,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.061091,"completed_at":"2026-06-14T15:02:05.294Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015023,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1613vm-test-run-nixbot-gitlab> gitlab # [  186.104401] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:05.366Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"78d22d8ebe03d3f27821b810","created_at":"2026-06-14T15:02:02.495Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7105847973913902c3e93f59bb9c5072","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:02.499Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-78d22d8ebe03d3f27821b810: start","job_status":"start","queue_duration_s":2.866572,"scheduling_latency_s":2.866635}1614vm-test-run-nixbot-gitlab> gitlab # [  186.156258] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:05.417Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"78d22d8ebe03d3f27821b810","created_at":"2026-06-14T15:02:02.495Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7105847973913902c3e93f59bb9c5072","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:02.499Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-78d22d8ebe03d3f27821b810: done: 0.051956 sec","job_status":"done","queue_duration_s":2.866572,"scheduling_latency_s":2.866635,"redis_calls":9,"redis_duration_s":0.003469,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001914,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001555,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019758,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001013257,"exclusive_lock_hold_duration_s":0.003691251,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.051956,"completed_at":"2026-06-14T15:02:05.417Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001406,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1615vm-test-run-nixbot-gitlab> gitlab # [  186.225641] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:05.487Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"993e320c07b30e92c363ce5e","created_at":"2026-06-14T15:02:02.524Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0422a69d2baed2fdee83d74e840ae91a","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:02.532Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-993e320c07b30e92c363ce5e: start","job_status":"start","queue_duration_s":2.954489,"scheduling_latency_s":2.954557}1616vm-test-run-nixbot-gitlab> gitlab # [  186.267946] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:05.529Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"993e320c07b30e92c363ce5e","created_at":"2026-06-14T15:02:02.524Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0422a69d2baed2fdee83d74e840ae91a","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:02.532Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-993e320c07b30e92c363ce5e: done: 0.04241 sec","job_status":"done","queue_duration_s":2.954489,"scheduling_latency_s":2.954557,"redis_calls":10,"redis_duration_s":0.006118,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003655,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002463,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025267,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.006527645,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.04241,"completed_at":"2026-06-14T15:02:05.529Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003236,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1617vm-test-run-nixbot-gitlab> gitlab # [  186.339410] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:05.601Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"e187e88fe448fdeef8952378","created_at":"2026-06-14T15:02:02.550Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"535465871e9fcff042ae96148f40644c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-14T15:02:02.554Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-e187e88fe448fdeef8952378: start","job_status":"start","queue_duration_s":3.046028,"scheduling_latency_s":3.046093}1618vm-test-run-nixbot-gitlab> gitlab # [  186.358863] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:05.620Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1619vm-test-run-nixbot-gitlab> gitlab # [  186.370911] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:05.632Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"e187e88fe448fdeef8952378","created_at":"2026-06-14T15:02:02.550Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"535465871e9fcff042ae96148f40644c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-14T15:02:02.554Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-e187e88fe448fdeef8952378: done: 0.031544 sec","job_status":"done","queue_duration_s":3.046028,"scheduling_latency_s":3.046093,"redis_calls":7,"redis_duration_s":0.0043300000000000005,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003209,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001121,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018497,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031544,"completed_at":"2026-06-14T15:02:05.632Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1620vm-test-run-nixbot-gitlab> gitlab # [  186.438289] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:05.697Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"bf753ea80ac14e479a359f84","created_at":"2026-06-14T15:02:02.566Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6a65536c85a5eff5887c4c14b66149af","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:02.571Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-bf753ea80ac14e479a359f84: start","job_status":"start","queue_duration_s":3.126223,"scheduling_latency_s":3.126279}1621vm-test-run-nixbot-gitlab> gitlab # [  186.519258] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:05.780Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"bf753ea80ac14e479a359f84","created_at":"2026-06-14T15:02:02.566Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6a65536c85a5eff5887c4c14b66149af","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:02.571Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-bf753ea80ac14e479a359f84: done: 0.083331 sec","job_status":"done","queue_duration_s":3.126223,"scheduling_latency_s":3.126279,"redis_calls":9,"redis_duration_s":0.005498,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00287,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002628,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02942,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001171937,"exclusive_lock_hold_duration_s":0.018071012,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.083331,"completed_at":"2026-06-14T15:02:05.780Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015739,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1622vm-test-run-nixbot-gitlab> gitlab # [  186.597816] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:05.859Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"1179c014d77f2835b296ed61","created_at":"2026-06-14T15:02:02.587Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d5637893588613753f3fdfa56a902007","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:02.597Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-1179c014d77f2835b296ed61: start","job_status":"start","queue_duration_s":3.262055,"scheduling_latency_s":3.262122}1623vm-test-run-nixbot-gitlab> gitlab # [  186.647872] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:05.909Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"1179c014d77f2835b296ed61","created_at":"2026-06-14T15:02:02.587Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d5637893588613753f3fdfa56a902007","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:02.597Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-1179c014d77f2835b296ed61: done: 0.049983 sec","job_status":"done","queue_duration_s":3.262055,"scheduling_latency_s":3.262122,"redis_calls":12,"redis_duration_s":0.00743,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001268,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003696,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002466,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027429,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001234236,"exclusive_lock_hold_duration_s":0.007633372,"duration_s":0.049983,"completed_at":"2026-06-14T15:02:05.909Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004545,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1624vm-test-run-nixbot-gitlab> gitlab # [  186.720087] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:05.981Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"df5bb3e4e6a739610fe39e40","created_at":"2026-06-14T15:02:02.607Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"40fb55149715e39fe31c3e6c04d3f9b5","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:02.617Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-df5bb3e4e6a739610fe39e40: start","job_status":"start","queue_duration_s":3.364667,"scheduling_latency_s":3.364732}1625vm-test-run-nixbot-gitlab> gitlab # [  186.958297] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:06.218Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"df5bb3e4e6a739610fe39e40","created_at":"2026-06-14T15:02:02.607Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-b01f4853c96342c5","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"40fb55149715e39fe31c3e6c04d3f9b5","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:02.617Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-df5bb3e4e6a739610fe39e40: done: 0.236904 sec","job_status":"done","queue_duration_s":3.364667,"scheduling_latency_s":3.364732,"redis_calls":10,"redis_duration_s":0.064754,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001185,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003343,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.060226,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035918,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000948445,"duration_s":0.236904,"completed_at":"2026-06-14T15:02:06.218Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006785,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1626vm-test-run-nixbot-gitlab> gitlab # [  187.037902] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:06.299Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fdabcde4759065dfe1a3f3d9","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fdabcde4759065dfe1a3f3d9: start","job_status":"start","queue_duration_s":2.634496,"scheduling_latency_s":2.634552}1627vm-test-run-nixbot-gitlab> gitlab # [  187.093855] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:06.354Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fdabcde4759065dfe1a3f3d9","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fdabcde4759065dfe1a3f3d9: done: 0.054791 sec","job_status":"done","queue_duration_s":2.634496,"scheduling_latency_s":2.634552,"redis_calls":13,"redis_duration_s":0.006805,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002128,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004677,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021186,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.054791,"completed_at":"2026-06-14T15:02:06.354Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1628vm-test-run-nixbot-gitlab> gitlab # [  187.159652] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:06.421Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8d8914ce11c95cd63193335c","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8d8914ce11c95cd63193335c: start","job_status":"start","queue_duration_s":2.755969,"scheduling_latency_s":2.756036}1629vm-test-run-nixbot-gitlab> gitlab # [  187.201499] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:06.463Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8d8914ce11c95cd63193335c","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8d8914ce11c95cd63193335c: done: 0.04192 sec","job_status":"done","queue_duration_s":2.755969,"scheduling_latency_s":2.756036,"redis_calls":13,"redis_duration_s":0.0076879999999999995,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002424,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005264,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025877,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.04192,"completed_at":"2026-06-14T15:02:06.462Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1630vm-test-run-nixbot-gitlab> gitlab # [  187.267360] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:06.528Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21327457e150413a1d752fab","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21327457e150413a1d752fab: start","job_status":"start","queue_duration_s":2.863556,"scheduling_latency_s":2.863621}1631vm-test-run-nixbot-gitlab> gitlab # [  187.308515] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:06.570Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21327457e150413a1d752fab","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21327457e150413a1d752fab: done: 0.041346 sec","job_status":"done","queue_duration_s":2.863556,"scheduling_latency_s":2.863621,"redis_calls":13,"redis_duration_s":0.008904,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002638,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006266,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025803,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.041346,"completed_at":"2026-06-14T15:02:06.569Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1632vm-test-run-nixbot-gitlab> gitlab # [  187.374195] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:06.635Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"592f73103d526c09605fadce","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-592f73103d526c09605fadce: start","job_status":"start","queue_duration_s":2.970478,"scheduling_latency_s":2.970562}1633vm-test-run-nixbot-gitlab> gitlab # [  187.434915] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:06.696Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"592f73103d526c09605fadce","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-592f73103d526c09605fadce: done: 0.060882 sec","job_status":"done","queue_duration_s":2.970478,"scheduling_latency_s":2.970562,"redis_calls":13,"redis_duration_s":0.006997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002859,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004138,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026164,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.060882,"completed_at":"2026-06-14T15:02:06.696Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1634vm-test-run-nixbot-gitlab> gitlab # [  187.504094] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:06.763Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5cf36cafce0c98beb1176a4a","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5cf36cafce0c98beb1176a4a: start","job_status":"start","queue_duration_s":3.097954,"scheduling_latency_s":3.098008}1635vm-test-run-nixbot-gitlab> gitlab # [  187.535664] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:06.797Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5cf36cafce0c98beb1176a4a","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5cf36cafce0c98beb1176a4a: done: 0.034667 sec","job_status":"done","queue_duration_s":3.097954,"scheduling_latency_s":3.098008,"redis_calls":13,"redis_duration_s":0.007181,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002205,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004976,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020422,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.034667,"completed_at":"2026-06-14T15:02:06.797Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1636vm-test-run-nixbot-gitlab> gitlab # [  187.573128] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:06.799Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f56338067053392f5957c78e","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f56338067053392f5957c78e: start","job_status":"start","queue_duration_s":3.134689,"scheduling_latency_s":3.134716}1637vm-test-run-nixbot-gitlab> gitlab # [  187.574322] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:06.818Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f56338067053392f5957c78e","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f56338067053392f5957c78e: done: 0.018274 sec","job_status":"done","queue_duration_s":3.134689,"scheduling_latency_s":3.134716,"redis_calls":13,"redis_duration_s":0.005578,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001572,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004006,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010444,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018274,"completed_at":"2026-06-14T15:02:06.818Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1638vm-test-run-nixbot-gitlab> gitlab # [  187.609795] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:06.819Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ec8bb8e8886ad300c41999b5","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ec8bb8e8886ad300c41999b5: start","job_status":"start","queue_duration_s":3.15475,"scheduling_latency_s":3.154771}1639vm-test-run-nixbot-gitlab> gitlab # [  187.705659] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:06.967Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ec8bb8e8886ad300c41999b5","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ec8bb8e8886ad300c41999b5: done: 0.147524 sec","job_status":"done","queue_duration_s":3.15475,"scheduling_latency_s":3.154771,"redis_calls":13,"redis_duration_s":0.13334700000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.129593,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003754,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011725,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.147524,"completed_at":"2026-06-14T15:02:06.967Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1640vm-test-run-nixbot-gitlab> gitlab # [  187.770763] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.032Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f30b22a6df063b641a9d5d8a","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f30b22a6df063b641a9d5d8a: start","job_status":"start","queue_duration_s":3.367116,"scheduling_latency_s":3.367181}1641vm-test-run-nixbot-gitlab> gitlab # [  187.810581] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.071Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f30b22a6df063b641a9d5d8a","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f30b22a6df063b641a9d5d8a: done: 0.038908 sec","job_status":"done","queue_duration_s":3.367116,"scheduling_latency_s":3.367181,"redis_calls":13,"redis_duration_s":0.007006,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002089,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004917,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023467,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.038908,"completed_at":"2026-06-14T15:02:07.071Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1642vm-test-run-nixbot-gitlab> gitlab # [  187.875486] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.137Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b4b652ddd04ebcfbda9ce8b","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b4b652ddd04ebcfbda9ce8b: start","job_status":"start","queue_duration_s":3.471817,"scheduling_latency_s":3.471904}1643vm-test-run-nixbot-gitlab> gitlab # [  187.933971] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.195Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b4b652ddd04ebcfbda9ce8b","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b4b652ddd04ebcfbda9ce8b: done: 0.058648 sec","job_status":"done","queue_duration_s":3.471817,"scheduling_latency_s":3.471904,"redis_calls":13,"redis_duration_s":0.008038,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002346,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005692,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026196,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.058648,"completed_at":"2026-06-14T15:02:07.195Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1644vm-test-run-nixbot-gitlab> gitlab # [  187.999845] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.260Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12c7f49a0bf084b43ed81b60","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12c7f49a0bf084b43ed81b60: start","job_status":"start","queue_duration_s":3.595549,"scheduling_latency_s":3.595614}1645vm-test-run-nixbot-gitlab> gitlab # [  188.038903] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.300Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12c7f49a0bf084b43ed81b60","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12c7f49a0bf084b43ed81b60: done: 0.03977 sec","job_status":"done","queue_duration_s":3.595549,"scheduling_latency_s":3.595614,"redis_calls":13,"redis_duration_s":0.007409,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002372,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005037,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024486,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.03977,"completed_at":"2026-06-14T15:02:07.300Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1646vm-test-run-nixbot-gitlab> gitlab # [  188.105123] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.365Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75fdc57dca260c0e006ccbf5","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75fdc57dca260c0e006ccbf5: start","job_status":"start","queue_duration_s":3.700573,"scheduling_latency_s":3.70064}1647vm-test-run-nixbot-gitlab> gitlab # [  188.146448] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.407Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75fdc57dca260c0e006ccbf5","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75fdc57dca260c0e006ccbf5: done: 0.042279 sec","job_status":"done","queue_duration_s":3.700573,"scheduling_latency_s":3.70064,"redis_calls":13,"redis_duration_s":0.00804,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002595,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005445,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02499,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.042279,"completed_at":"2026-06-14T15:02:07.407Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1648vm-test-run-nixbot-gitlab> gitlab # [  188.211443] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.473Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1119b2c644b024879d61b112","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1119b2c644b024879d61b112: start","job_status":"start","queue_duration_s":3.807797,"scheduling_latency_s":3.807881}1649vm-test-run-nixbot-gitlab> gitlab # [  188.253107] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.514Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1119b2c644b024879d61b112","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1119b2c644b024879d61b112: done: 0.041582 sec","job_status":"done","queue_duration_s":3.807797,"scheduling_latency_s":3.807881,"redis_calls":13,"redis_duration_s":0.008316,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002505,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005811,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025966,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.041582,"completed_at":"2026-06-14T15:02:07.514Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1650vm-test-run-nixbot-gitlab> gitlab # [  188.318840] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.580Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a25effefb1fdba1cf7dc400f","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a25effefb1fdba1cf7dc400f: start","job_status":"start","queue_duration_s":3.915192,"scheduling_latency_s":3.915351}1651vm-test-run-nixbot-gitlab> gitlab # [  188.358946] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.620Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a25effefb1fdba1cf7dc400f","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a25effefb1fdba1cf7dc400f: done: 0.039949 sec","job_status":"done","queue_duration_s":3.915192,"scheduling_latency_s":3.915351,"redis_calls":13,"redis_duration_s":0.007308,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002337,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004971,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024677,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.039949,"completed_at":"2026-06-14T15:02:07.620Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1652vm-test-run-nixbot-gitlab> gitlab # [  188.361989] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.623Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"584b4204c4436ceed409750c","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-584b4204c4436ceed409750c: start","job_status":"start","queue_duration_s":3.95866,"scheduling_latency_s":3.958714}1653vm-test-run-nixbot-gitlab> gitlab # [  188.395591] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.657Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"584b4204c4436ceed409750c","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-584b4204c4436ceed409750c: done: 0.033444 sec","job_status":"done","queue_duration_s":3.95866,"scheduling_latency_s":3.958714,"redis_calls":13,"redis_duration_s":0.006574,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002063,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004511,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019024,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.033444,"completed_at":"2026-06-14T15:02:07.657Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1654vm-test-run-nixbot-gitlab> gitlab # [  188.436152] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.660Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d8e228f01d9b4516db2c2531","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d8e228f01d9b4516db2c2531: start","job_status":"start","queue_duration_s":3.994904,"scheduling_latency_s":3.994929}1655vm-test-run-nixbot-gitlab> gitlab # [  188.456212] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.675Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d8e228f01d9b4516db2c2531","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d8e228f01d9b4516db2c2531: done: 0.015494 sec","job_status":"done","queue_duration_s":3.994904,"scheduling_latency_s":3.994929,"redis_calls":13,"redis_duration_s":0.0030099999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000936,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002074,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009901,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015494,"completed_at":"2026-06-14T15:02:07.675Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1656vm-test-run-nixbot-gitlab> gitlab # [  188.530979] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.676Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3f7a21747870495d59f3d51","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3f7a21747870495d59f3d51: start","job_status":"start","queue_duration_s":4.011686,"scheduling_latency_s":4.011707}1657vm-test-run-nixbot-gitlab> gitlab # [  188.537614] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.692Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3f7a21747870495d59f3d51","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3f7a21747870495d59f3d51: done: 0.015316 sec","job_status":"done","queue_duration_s":4.011686,"scheduling_latency_s":4.011707,"redis_calls":13,"redis_duration_s":0.003404,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00094,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002464,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009705,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015316,"completed_at":"2026-06-14T15:02:07.692Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1658vm-test-run-nixbot-gitlab> gitlab # [  188.615304] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.693Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3615f479d952898b54579a09","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3615f479d952898b54579a09: start","job_status":"start","queue_duration_s":4.02833,"scheduling_latency_s":4.028352}1659vm-test-run-nixbot-gitlab> gitlab # [  188.619762] runuser[2669]: pam_unix(runuser:session): session closed for user gitlab1660vm-test-run-nixbot-gitlab> gitlab # [  188.621786] setup-gitlab-start[2893]: ++ jq -r .token1661vm-test-run-nixbot-gitlab> gitlab # [  188.623061] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.709Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3615f479d952898b54579a09","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3615f479d952898b54579a09: done: 0.015514 sec","job_status":"done","queue_duration_s":4.02833,"scheduling_latency_s":4.028352,"redis_calls":13,"redis_duration_s":0.002986,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000961,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002025,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009651,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015514,"completed_at":"2026-06-14T15:02:07.709Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1662vm-test-run-nixbot-gitlab> gitlab # [  188.716260] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.710Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9514f31c3cc9e72084857cdd","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9514f31c3cc9e72084857cdd: start","job_status":"start","queue_duration_s":4.045115,"scheduling_latency_s":4.045137}1663vm-test-run-nixbot-gitlab> gitlab # [  188.718049] setup-gitlab-start[2894]: +++ date -d '+7 days' +%F1664vm-test-run-nixbot-gitlab> gitlab # [  188.719074] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.732Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9514f31c3cc9e72084857cdd","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9514f31c3cc9e72084857cdd: done: 0.02257 sec","job_status":"done","queue_duration_s":4.045115,"scheduling_latency_s":4.045137,"redis_calls":13,"redis_duration_s":0.002941,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00093,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002011,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009688,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02257,"completed_at":"2026-06-14T15:02:07.732Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1665vm-test-run-nixbot-gitlab> gitlab # [  188.794376] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.734Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc3a446e9a8622d8bf71509a","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc3a446e9a8622d8bf71509a: start","job_status":"start","queue_duration_s":4.068987,"scheduling_latency_s":4.069008}1666vm-test-run-nixbot-gitlab> gitlab # [  188.795953] setup-gitlab-start[2892]: ++ curl -fs -X POST -H 'Authorization: Bearer 9857042119ca9ef6839634ae0326d9ed1fcf9afc058fdf9a40aa22e4e0a4d566' -H 'Content-Type: application/json' -d '{"name":"nixbot","scopes":["api"],"expires_at":"2026-06-21"}' http://gitlab/api/v4/users/1/personal_access_tokens1667vm-test-run-nixbot-gitlab> gitlab # [  188.797506] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:08 +0000] "POST /api/v4/users/1/personal_access_tokens HTTP/1.1" 201 267 "" "curl/8.20.0" 2911668vm-test-run-nixbot-gitlab> gitlab # [  188.798429] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.752Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc3a446e9a8622d8bf71509a","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc3a446e9a8622d8bf71509a: done: 0.018141 sec","job_status":"done","queue_duration_s":4.068987,"scheduling_latency_s":4.069008,"redis_calls":13,"redis_duration_s":0.003358,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000904,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002454,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011431,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018141,"completed_at":"2026-06-14T15:02:07.752Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1669vm-test-run-nixbot-gitlab> gitlab # [  188.877692] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.753Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad956cd92f16ad3439823ecd","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad956cd92f16ad3439823ecd: start","job_status":"start","queue_duration_s":4.0886,"scheduling_latency_s":4.088621}1670vm-test-run-nixbot-gitlab> gitlab # [  188.879162] setup-gitlab-start[2618]: + TOKEN=glpat-T9cZOZ5lxCGBJ6y2CDCkeG86MQp1OjEH.01.0w0mp5g2p1671vm-test-run-nixbot-gitlab> gitlab # [  188.879935] setup-gitlab-start[2618]: + test -n glpat-T9cZOZ5lxCGBJ6y2CDCkeG86MQp1OjEH.01.0w0mp5g2p1672vm-test-run-nixbot-gitlab> gitlab # [  188.881952] setup-gitlab-start[2618]: + test glpat-T9cZOZ5lxCGBJ6y2CDCkeG86MQp1OjEH.01.0w0mp5g2p '!=' null1673vm-test-run-nixbot-gitlab> gitlab # [  188.883689] setup-gitlab-start[2618]: + mkdir -p /var/lib/secrets1674vm-test-run-nixbot-gitlab> gitlab # [  188.885158] setup-gitlab-start[2618]: + echo glpat-T9cZOZ5lxCGBJ6y2CDCkeG86MQp1OjEH.01.0w0mp5g2p1675vm-test-run-nixbot-gitlab> gitlab # [  188.887182] setup-gitlab-start[2618]: + chmod 644 /var/lib/secrets/gitlab-token1676vm-test-run-nixbot-gitlab> gitlab # [  188.888304] setup-gitlab-start[2618]: + curl -fs -X POST -H 'PRIVATE-TOKEN: glpat-T9cZOZ5lxCGBJ6y2CDCkeG86MQp1OjEH.01.0w0mp5g2p' -H 'Content-Type: application/json' -d '{"name":"test-flake","visibility":"public","topics":["build-with-buildbot"]}' http://gitlab/api/v4/projects1677vm-test-run-nixbot-gitlab> gitlab # [  188.957444] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.769Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad956cd92f16ad3439823ecd","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad956cd92f16ad3439823ecd: done: 0.015425 sec","job_status":"done","queue_duration_s":4.0886,"scheduling_latency_s":4.088621,"redis_calls":13,"redis_duration_s":0.003028,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000971,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002057,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009608,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015425,"completed_at":"2026-06-14T15:02:07.769Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1678vm-test-run-nixbot-gitlab> gitlab # [  188.959113] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.770Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"93dfb5ee79f3a8a82ab91cef","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-93dfb5ee79f3a8a82ab91cef: start","job_status":"start","queue_duration_s":4.105341,"scheduling_latency_s":4.105384}1679vm-test-run-nixbot-gitlab> gitlab # [  188.974863] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.785Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"93dfb5ee79f3a8a82ab91cef","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-93dfb5ee79f3a8a82ab91cef: done: 0.015415 sec","job_status":"done","queue_duration_s":4.105341,"scheduling_latency_s":4.105384,"redis_calls":13,"redis_duration_s":0.003124,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000928,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002196,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009306,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015415,"completed_at":"2026-06-14T15:02:07.785Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1680vm-test-run-nixbot-gitlab> gitlab # [  189.051423] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.787Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5be6ed12794a75db50f30921","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5be6ed12794a75db50f30921: start","job_status":"start","queue_duration_s":4.122553,"scheduling_latency_s":4.122574}1681vm-test-run-nixbot-gitlab> gitlab # [  189.123204] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.802Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5be6ed12794a75db50f30921","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5be6ed12794a75db50f30921: done: 0.015136 sec","job_status":"done","queue_duration_s":4.122553,"scheduling_latency_s":4.122574,"redis_calls":13,"redis_duration_s":0.0029189999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000884,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002035,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009789,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015136,"completed_at":"2026-06-14T15:02:07.802Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1682vm-test-run-nixbot-gitlab> gitlab # [  189.125091] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.804Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a2e432b291251a730f777cb4","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a2e432b291251a730f777cb4: start","job_status":"start","queue_duration_s":4.138963,"scheduling_latency_s":4.138984}1683vm-test-run-nixbot-gitlab> gitlab # [  189.206005] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.819Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a2e432b291251a730f777cb4","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a2e432b291251a730f777cb4: done: 0.014957 sec","job_status":"done","queue_duration_s":4.138963,"scheduling_latency_s":4.138984,"redis_calls":13,"redis_duration_s":0.0029040000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000946,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001958,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009522,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014957,"completed_at":"2026-06-14T15:02:07.819Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1684vm-test-run-nixbot-gitlab> gitlab # [  189.208108] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.820Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86ae7bd9b0ddf8f4dd93e6dd","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86ae7bd9b0ddf8f4dd93e6dd: start","job_status":"start","queue_duration_s":4.155332,"scheduling_latency_s":4.155353}1685vm-test-run-nixbot-gitlab> gitlab # [  189.283557] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.835Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86ae7bd9b0ddf8f4dd93e6dd","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86ae7bd9b0ddf8f4dd93e6dd: done: 0.01475 sec","job_status":"done","queue_duration_s":4.155332,"scheduling_latency_s":4.155353,"redis_calls":13,"redis_duration_s":0.00278,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000937,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001843,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009578,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01475,"completed_at":"2026-06-14T15:02:07.835Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1686vm-test-run-nixbot-gitlab> gitlab # [  189.285712] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.836Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"61372a2a6d3c53034ff14d53","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-61372a2a6d3c53034ff14d53: start","job_status":"start","queue_duration_s":4.171736,"scheduling_latency_s":4.171757}1687vm-test-run-nixbot-gitlab> gitlab # [  189.360398] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.851Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"61372a2a6d3c53034ff14d53","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-61372a2a6d3c53034ff14d53: done: 0.014712 sec","job_status":"done","queue_duration_s":4.171736,"scheduling_latency_s":4.171757,"redis_calls":13,"redis_duration_s":0.002715,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000844,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001871,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010041,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014712,"completed_at":"2026-06-14T15:02:07.851Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1688vm-test-run-nixbot-gitlab> gitlab # [  189.431925] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.852Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34c933fdda2355faa99a0476","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34c933fdda2355faa99a0476: start","job_status":"start","queue_duration_s":4.187664,"scheduling_latency_s":4.187684}1689vm-test-run-nixbot-gitlab> gitlab # [  189.433128] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.867Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34c933fdda2355faa99a0476","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34c933fdda2355faa99a0476: done: 0.014586 sec","job_status":"done","queue_duration_s":4.187664,"scheduling_latency_s":4.187684,"redis_calls":13,"redis_duration_s":0.003089,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000811,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002278,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009963,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014586,"completed_at":"2026-06-14T15:02:07.867Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1690vm-test-run-nixbot-gitlab> gitlab # [  189.451268] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.868Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cbca3b9526618977f01d75c0","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cbca3b9526618977f01d75c0: start","job_status":"start","queue_duration_s":4.203493,"scheduling_latency_s":4.203514}1691vm-test-run-nixbot-gitlab> gitlab # [  189.530427] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.883Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cbca3b9526618977f01d75c0","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cbca3b9526618977f01d75c0: done: 0.015079 sec","job_status":"done","queue_duration_s":4.203493,"scheduling_latency_s":4.203514,"redis_calls":13,"redis_duration_s":0.003209,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000892,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002317,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009778,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015079,"completed_at":"2026-06-14T15:02:07.883Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1692vm-test-run-nixbot-gitlab> gitlab # [  189.534341] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.885Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ccebdec34a82a0abf7f662e5","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ccebdec34a82a0abf7f662e5: start","job_status":"start","queue_duration_s":4.219946,"scheduling_latency_s":4.219969}1693vm-test-run-nixbot-gitlab> gitlab # [  189.604991] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.900Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ccebdec34a82a0abf7f662e5","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ccebdec34a82a0abf7f662e5: done: 0.015592 sec","job_status":"done","queue_duration_s":4.219946,"scheduling_latency_s":4.219969,"redis_calls":13,"redis_duration_s":0.003074,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000917,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002157,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009213,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015592,"completed_at":"2026-06-14T15:02:07.900Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1694vm-test-run-nixbot-gitlab> gitlab # [  189.676840] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.902Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0374a6177d082015c6ee9f08","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0374a6177d082015c6ee9f08: start","job_status":"start","queue_duration_s":4.236841,"scheduling_latency_s":4.236863}1695vm-test-run-nixbot-gitlab> gitlab # [  189.678239] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.917Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0374a6177d082015c6ee9f08","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0374a6177d082015c6ee9f08: done: 0.015226 sec","job_status":"done","queue_duration_s":4.236841,"scheduling_latency_s":4.236863,"redis_calls":13,"redis_duration_s":0.003033,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001014,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002019,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009775,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015226,"completed_at":"2026-06-14T15:02:07.917Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1696vm-test-run-nixbot-gitlab> gitlab # [  189.749894] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.918Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9dcd84c9ce9c99631d32d6ce","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9dcd84c9ce9c99631d32d6ce: start","job_status":"start","queue_duration_s":4.253407,"scheduling_latency_s":4.253428}1697vm-test-run-nixbot-gitlab> gitlab # [  189.751053] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.933Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9dcd84c9ce9c99631d32d6ce","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9dcd84c9ce9c99631d32d6ce: done: 0.015372 sec","job_status":"done","queue_duration_s":4.253407,"scheduling_latency_s":4.253428,"redis_calls":13,"redis_duration_s":0.0035570000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001423,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002134,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009993,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015372,"completed_at":"2026-06-14T15:02:07.933Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1698vm-test-run-nixbot-gitlab> gitlab # [  189.828864] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.935Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7c912cd9b480a1675a72cee","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7c912cd9b480a1675a72cee: start","job_status":"start","queue_duration_s":4.270066,"scheduling_latency_s":4.270087}1699vm-test-run-nixbot-gitlab> gitlab # [  189.830339] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.950Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7c912cd9b480a1675a72cee","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7c912cd9b480a1675a72cee: done: 0.015099 sec","job_status":"done","queue_duration_s":4.270066,"scheduling_latency_s":4.270087,"redis_calls":13,"redis_duration_s":0.002914,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000912,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002002,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009254,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015099,"completed_at":"2026-06-14T15:02:07.950Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1700vm-test-run-nixbot-gitlab> gitlab # [  189.908817] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.952Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"72654baa0a27bd47ddf4c9db","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-72654baa0a27bd47ddf4c9db: start","job_status":"start","queue_duration_s":4.28705,"scheduling_latency_s":4.287071}1701vm-test-run-nixbot-gitlab> gitlab # [  189.910289] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.967Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"72654baa0a27bd47ddf4c9db","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-72654baa0a27bd47ddf4c9db: done: 0.015348 sec","job_status":"done","queue_duration_s":4.28705,"scheduling_latency_s":4.287071,"redis_calls":13,"redis_duration_s":0.0028929999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000861,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002032,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009985,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015348,"completed_at":"2026-06-14T15:02:07.967Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1702vm-test-run-nixbot-gitlab> gitlab # [  189.980253] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.968Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f97779f057094b597ba7ac8","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f97779f057094b597ba7ac8: start","job_status":"start","queue_duration_s":4.303635,"scheduling_latency_s":4.303657}1703vm-test-run-nixbot-gitlab> gitlab # [  189.981883] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.985Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f97779f057094b597ba7ac8","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f97779f057094b597ba7ac8: done: 0.016213 sec","job_status":"done","queue_duration_s":4.303635,"scheduling_latency_s":4.303657,"redis_calls":13,"redis_duration_s":0.003908,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00096,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002948,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010084,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016213,"completed_at":"2026-06-14T15:02:07.985Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1704vm-test-run-nixbot-gitlab> gitlab # [  190.073588] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:07.987Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f7a160c3f9ecc73cf1fcdc0","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f7a160c3f9ecc73cf1fcdc0: start","job_status":"start","queue_duration_s":4.322562,"scheduling_latency_s":4.3226}1705vm-test-run-nixbot-gitlab> gitlab # [  190.075053] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.006Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f7a160c3f9ecc73cf1fcdc0","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f7a160c3f9ecc73cf1fcdc0: done: 0.018599 sec","job_status":"done","queue_duration_s":4.322562,"scheduling_latency_s":4.3226,"redis_calls":13,"redis_duration_s":0.003001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000933,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002068,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01104,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018599,"completed_at":"2026-06-14T15:02:08.006Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1706vm-test-run-nixbot-gitlab> gitlab # [  190.144669] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.007Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c8aa8d938b9bb0cebe9c445","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c8aa8d938b9bb0cebe9c445: start","job_status":"start","queue_duration_s":4.342433,"scheduling_latency_s":4.342454}1707vm-test-run-nixbot-gitlab> gitlab # [  190.145854] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.024Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c8aa8d938b9bb0cebe9c445","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c8aa8d938b9bb0cebe9c445: done: 0.01649 sec","job_status":"done","queue_duration_s":4.342433,"scheduling_latency_s":4.342454,"redis_calls":13,"redis_duration_s":0.002886,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000853,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002033,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010578,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01649,"completed_at":"2026-06-14T15:02:08.024Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1708vm-test-run-nixbot-gitlab> gitlab # [  190.216944] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.025Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0f587af17294bcb819ec83d8","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0f587af17294bcb819ec83d8: start","job_status":"start","queue_duration_s":4.360441,"scheduling_latency_s":4.360466}1709vm-test-run-nixbot-gitlab> gitlab # [  190.218184] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.041Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0f587af17294bcb819ec83d8","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0f587af17294bcb819ec83d8: done: 0.015957 sec","job_status":"done","queue_duration_s":4.360441,"scheduling_latency_s":4.360466,"redis_calls":13,"redis_duration_s":0.0029189999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001018,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001901,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010759,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015957,"completed_at":"2026-06-14T15:02:08.041Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1710vm-test-run-nixbot-gitlab> gitlab # [  190.308821] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.042Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"272c1114e95e740623ed271a","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-272c1114e95e740623ed271a: start","job_status":"start","queue_duration_s":4.377737,"scheduling_latency_s":4.37776}1711vm-test-run-nixbot-gitlab> gitlab # [  190.310354] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.061Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"272c1114e95e740623ed271a","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-272c1114e95e740623ed271a: done: 0.018933 sec","job_status":"done","queue_duration_s":4.377737,"scheduling_latency_s":4.37776,"redis_calls":13,"redis_duration_s":0.003619,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00093,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002689,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012758,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018933,"completed_at":"2026-06-14T15:02:08.061Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1712vm-test-run-nixbot-gitlab> gitlab # [  190.389900] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.063Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"636bde3a83104d5e6616df6c","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-636bde3a83104d5e6616df6c: start","job_status":"start","queue_duration_s":4.398131,"scheduling_latency_s":4.398155}1713vm-test-run-nixbot-gitlab> gitlab # [  190.392762] gitaly[1387]: time="2026-06-14T15:02:09.486Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KV3AF1DN5Y5RCD6J8RHJZRC8 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-14T15:02:19.005" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T15:02:09.486" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1714vm-test-run-nixbot-gitlab> gitlab # [  190.398269] gitaly[1387]: time="2026-06-14T15:02:09.486Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KV3AF1DN5Y5RCD6J8RHJZRC8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-14T15:02:19.005" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T15:02:09.486" grpc.time_ms=0.347 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1715vm-test-run-nixbot-gitlab> gitlab # [  190.474298] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.079Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"636bde3a83104d5e6616df6c","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-636bde3a83104d5e6616df6c: done: 0.016298 sec","job_status":"done","queue_duration_s":4.398131,"scheduling_latency_s":4.398155,"redis_calls":13,"redis_duration_s":0.003084,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000926,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002158,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010278,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016298,"completed_at":"2026-06-14T15:02:08.079Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1716vm-test-run-nixbot-gitlab> gitlab # [  190.482038] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.081Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53cac38dfbb361d149806ab1","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53cac38dfbb361d149806ab1: start","job_status":"start","queue_duration_s":4.416224,"scheduling_latency_s":4.416283}1717vm-test-run-nixbot-gitlab> gitlab # [  190.502855] gitaly[1387]: time="2026-06-14T15:02:09.568Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KV3AF1DN5Y5RCD6J8RHJZRC8 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-14T15:02:19.087" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T15:02:09.568" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1718vm-test-run-nixbot-gitlab> gitlab # [  190.573949] gitaly[1387]: time="2026-06-14T15:02:09.568Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KV3AF1DN5Y5RCD6J8RHJZRC8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-14T15:02:19.087" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T15:02:09.568" grpc.time_ms=0.266 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1719vm-test-run-nixbot-gitlab> gitlab # [  190.575126] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.097Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53cac38dfbb361d149806ab1","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53cac38dfbb361d149806ab1: done: 0.015964 sec","job_status":"done","queue_duration_s":4.416224,"scheduling_latency_s":4.416283,"redis_calls":13,"redis_duration_s":0.002875,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000888,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001987,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010904,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015964,"completed_at":"2026-06-14T15:02:08.097Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1720vm-test-run-nixbot-gitlab> gitlab # [  190.648207] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.098Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"538a6112d324e35a53f2e3cf","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-538a6112d324e35a53f2e3cf: start","job_status":"start","queue_duration_s":4.433632,"scheduling_latency_s":4.433654}1721vm-test-run-nixbot-gitlab> gitlab # [  190.651323] gitaly[1387]: time="2026-06-14T15:02:09.606Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KV3AF1DN5Y5RCD6J8RHJZRC8 diskcache=69b006dd-deaa-4240-afb3-044292fc1e34 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-14T15:02:19.096" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T15:02:09.576" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1722vm-test-run-nixbot-gitlab> gitlab # [  190.653852] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.114Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"538a6112d324e35a53f2e3cf","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-538a6112d324e35a53f2e3cf: done: 0.015492 sec","job_status":"done","queue_duration_s":4.433632,"scheduling_latency_s":4.433654,"redis_calls":13,"redis_duration_s":0.002843,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000886,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001957,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010395,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015492,"completed_at":"2026-06-14T15:02:08.114Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1723vm-test-run-nixbot-gitlab> gitlab # [  190.732401] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.115Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8e66efc3d47072f6620ff8a4","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8e66efc3d47072f6620ff8a4: start","job_status":"start","queue_duration_s":4.450347,"scheduling_latency_s":4.450368}1724vm-test-run-nixbot-gitlab> gitlab # [  190.734062] gitaly[1387]: time="2026-06-14T15:02:09.606Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=329 command.oublock=32 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AF1DN5Y5RCD6J8RHJZRC8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-14T15:02:19.096" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T15:02:09.576" grpc.time_ms=30.093 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1725vm-test-run-nixbot-gitlab> gitlab # [  190.804415] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.130Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8e66efc3d47072f6620ff8a4","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8e66efc3d47072f6620ff8a4: done: 0.014644 sec","job_status":"done","queue_duration_s":4.450347,"scheduling_latency_s":4.450368,"redis_calls":13,"redis_duration_s":0.002683,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000847,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001836,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009682,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014644,"completed_at":"2026-06-14T15:02:08.130Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1726vm-test-run-nixbot-gitlab> gitlab # [  190.806656] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.131Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"272d125312ebe17df2f6c0ad","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-272d125312ebe17df2f6c0ad: start","job_status":"start","queue_duration_s":4.466547,"scheduling_latency_s":4.466567}1727vm-test-run-nixbot-gitlab> gitlab # [  190.880189] gitaly[1387]: time="2026-06-14T15:02:09.615Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KV3AF1DN5Y5RCD6J8RHJZRC8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-14T15:02:19.134" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T15:02:09.614" pid=1387 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1728vm-test-run-nixbot-gitlab> gitlab # [  190.881809] gitaly[1387]: time="2026-06-14T15:02:09.615Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KV3AF1DN5Y5RCD6J8RHJZRC8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-14T15:02:19.134" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T15:02:09.614" grpc.time_ms=0.367 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1729vm-test-run-nixbot-gitlab> gitlab # [  190.883072] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.146Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"272d125312ebe17df2f6c0ad","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-272d125312ebe17df2f6c0ad: done: 0.014961 sec","job_status":"done","queue_duration_s":4.466547,"scheduling_latency_s":4.466567,"redis_calls":13,"redis_duration_s":0.002973,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002073,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009751,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014961,"completed_at":"2026-06-14T15:02:08.146Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1730vm-test-run-nixbot-gitlab> gitlab # [  190.962882] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.148Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"2b3f7854fb17d55b347eeb99","created_at":"2026-06-14T15:02:08.004Z","trace_propagation_headers":{"sentry-trace":"f44c1b3c364443b48b8813968e1f9412-20b75bdf464041f6","baggage":"sentry-trace_id=f44c1b3c364443b48b8813968e1f9412,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV3AF141KPARZTDRDN91VDP7","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:ec79542482632c5ce9f79bb038a3b15f8e8ac74c733b2f297de7562da91ce3f5","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:08.030Z","job_size_bytes":451,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-2b3f7854fb17d55b347eeb99: start","job_status":"start","queue_duration_s":0.117709,"scheduling_latency_s":0.117732}1731vm-test-run-nixbot-gitlab> gitlab # [  190.964262] gitaly[1387]: time="2026-06-14T15:02:09.623Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AF1DN5Y5RCD6J8RHJZRC8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-14T15:02:19.137" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T15:02:09.618" grpc.time_ms=5.259 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1732vm-test-run-nixbot-gitlab> gitlab # [  190.966852] gitaly[1387]: time="2026-06-14T15:02:09.654Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KV3AF1DN5Y5RCD6J8RHJZRC8 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-14T15:02:19.173" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T15:02:09.653" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1733vm-test-run-nixbot-gitlab> gitlab # [  191.054470] gitaly[1387]: time="2026-06-14T15:02:09.654Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KV3AF1DN5Y5RCD6J8RHJZRC8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-14T15:02:19.173" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T15:02:09.653" grpc.time_ms=0.238 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1734vm-test-run-nixbot-gitlab> gitlab # [  191.056537] setup-gitlab-start[2618]: + mkdir -p /tmp/test-flake1735vm-test-run-nixbot-gitlab> gitlab # [  191.057381] setup-gitlab-start[2618]: + cd /tmp/test-flake1736vm-test-run-nixbot-gitlab> gitlab # [  191.058295] setup-gitlab-start[2618]: + git init -b master1737vm-test-run-nixbot-gitlab> gitlab # [  191.059274] gitaly[1387]: time="2026-06-14T15:02:09.684Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KV3AF1DN5Y5RCD6J8RHJZRC8 diskcache=2204f148-d090-4b0d-8c05-2ec154d0f0e2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-14T15:02:19.178" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T15:02:09.658" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1738vm-test-run-nixbot-gitlab> gitlab # [  191.060432] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:10 +0000] "POST /api/v4/projects HTTP/1.1" 201 4795 "" "curl/8.20.0" 21941739vm-test-run-nixbot-gitlab> gitlab # [  191.061610] setup-gitlab-start[2938]: Initialized empty Git repository in /tmp/test-flake/.git/1740vm-test-run-nixbot-gitlab> gitlab # [  191.062909] gitaly[1387]: time="2026-06-14T15:02:09.684Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=328 command.oublock=32 command.real_time_ms=4 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AF1DN5Y5RCD6J8RHJZRC8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-14T15:02:19.178" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T15:02:09.658" grpc.time_ms=25.352 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1741vm-test-run-nixbot-gitlab> gitlab # [  191.064235] sidekiq-cluster[2097]: {"severity":"WARN","time":"2026-06-14T15:02:08.219Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"2b3f7854fb17d55b347eeb99","created_at":"2026-06-14T15:02:08.004Z","trace_propagation_headers":{"sentry-trace":"f44c1b3c364443b48b8813968e1f9412-20b75bdf464041f6","baggage":"sentry-trace_id=f44c1b3c364443b48b8813968e1f9412,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV3AF141KPARZTDRDN91VDP7","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:ec79542482632c5ce9f79bb038a3b15f8e8ac74c733b2f297de7562da91ce3f5","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:08.030Z","job_size_bytes":451,"pid":2097,"sidekiq_tid":"lyr9","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-2b3f7854fb17d55b347eeb99: fail: 0.070722 sec","job_status":"fail","queue_duration_s":0.117709,"scheduling_latency_s":0.117732,"redis_calls":11,"redis_duration_s":0.00246,"redis_read_bytes":137,"redis_write_bytes":1317,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000657,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001384,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000419,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053646,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000341385,"duration_s":0.070722,"completed_at":"2026-06-14T15:02:08.218Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.009371,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1742vm-test-run-nixbot-gitlab> gitlab # [  191.692701] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.222Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e39ca2d84deacbbecdde0f18","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e39ca2d84deacbbecdde0f18: start","job_status":"start","queue_duration_s":4.557203,"scheduling_latency_s":4.557228}1743vm-test-run-nixbot-gitlab> gitlab # [  191.771425] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.238Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e39ca2d84deacbbecdde0f18","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e39ca2d84deacbbecdde0f18: done: 0.015798 sec","job_status":"done","queue_duration_s":4.557203,"scheduling_latency_s":4.557228,"redis_calls":13,"redis_duration_s":0.002924,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000922,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002002,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009836,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015798,"completed_at":"2026-06-14T15:02:08.238Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1744vm-test-run-nixbot-gitlab> gitlab # [  191.773141] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.239Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c460c0cc341843317fca4fd0","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c460c0cc341843317fca4fd0: start","job_status":"start","queue_duration_s":4.574305,"scheduling_latency_s":4.574327}1745vm-test-run-nixbot-gitlab> gitlab # [  191.861093] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.254Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c460c0cc341843317fca4fd0","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c460c0cc341843317fca4fd0: done: 0.014735 sec","job_status":"done","queue_duration_s":4.574305,"scheduling_latency_s":4.574327,"redis_calls":13,"redis_duration_s":0.002657,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000849,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001808,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009832,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014735,"completed_at":"2026-06-14T15:02:08.254Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1746vm-test-run-nixbot-gitlab> gitlab # [  191.862394] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.255Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b56e2a6b53573078fbb6d6e2","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b56e2a6b53573078fbb6d6e2: start","job_status":"start","queue_duration_s":4.590198,"scheduling_latency_s":4.590218}1747vm-test-run-nixbot-gitlab> gitlab # [  191.938187] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.270Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b56e2a6b53573078fbb6d6e2","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b56e2a6b53573078fbb6d6e2: done: 0.014671 sec","job_status":"done","queue_duration_s":4.590198,"scheduling_latency_s":4.590218,"redis_calls":13,"redis_duration_s":0.002522,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000795,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001727,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009979,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014671,"completed_at":"2026-06-14T15:02:08.270Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1748vm-test-run-nixbot-gitlab> gitlab # [  191.940547] setup-gitlab-start[2618]: + git config user.name test1749vm-test-run-nixbot-gitlab> gitlab # [  191.942126] setup-gitlab-start[2618]: + git config user.email test@example.com1750vm-test-run-nixbot-gitlab> gitlab # [  191.943304] setup-gitlab-start[2618]: + cat1751vm-test-run-nixbot-gitlab> gitlab # [  191.944199] setup-gitlab-start[2618]: + git add flake.nix1752vm-test-run-nixbot-gitlab> gitlab # [  192.014140] setup-gitlab-start[2618]: + git commit -m 'initial commit'1753vm-test-run-nixbot-gitlab> gitlab # [  192.015641] gitaly[1387]: time="2026-06-14T15:02:09.690Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KV3AF1DN5Y5RCD6J8RHJZRC8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-14T15:02:19.209" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T15:02:09.689" pid=1387 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1754vm-test-run-nixbot-gitlab> gitlab # [  192.016747] gitaly[1387]: time="2026-06-14T15:02:09.690Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KV3AF1DN5Y5RCD6J8RHJZRC8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-14T15:02:19.209" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T15:02:09.689" grpc.time_ms=0.346 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1755vm-test-run-nixbot-gitlab> gitlab # [  192.017945] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:10 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 1641756vm-test-run-nixbot-gitlab> gitlab # [  192.019058] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:10 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.54.0" 1371757vm-test-run-nixbot-gitlab> gitlab # [  192.020059] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.271Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04430eb1fc26e5c3ab625448","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04430eb1fc26e5c3ab625448: start","job_status":"start","queue_duration_s":4.60606,"scheduling_latency_s":4.60608}1758vm-test-run-nixbot-gitlab> gitlab # [  192.020865] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.285Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04430eb1fc26e5c3ab625448","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04430eb1fc26e5c3ab625448: done: 0.014698 sec","job_status":"done","queue_duration_s":4.60606,"scheduling_latency_s":4.60608,"redis_calls":13,"redis_duration_s":0.002829,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000984,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001845,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009643,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014698,"completed_at":"2026-06-14T15:02:08.285Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1759vm-test-run-nixbot-gitlab> gitlab # [  192.094189] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.287Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"10daadfe46eff5b7876d76ac","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-10daadfe46eff5b7876d76ac: start","job_status":"start","queue_duration_s":4.622136,"scheduling_latency_s":4.622158}1760vm-test-run-nixbot-gitlab> gitlab # [  192.095776] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.301Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"10daadfe46eff5b7876d76ac","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-10daadfe46eff5b7876d76ac: done: 0.014564 sec","job_status":"done","queue_duration_s":4.622136,"scheduling_latency_s":4.622158,"redis_calls":13,"redis_duration_s":0.002662,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000792,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00187,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009836,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014564,"completed_at":"2026-06-14T15:02:08.301Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1761vm-test-run-nixbot-gitlab> gitlab # [  192.195861] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.303Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dadbad6769e0d587f70f318f","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dadbad6769e0d587f70f318f: start","job_status":"start","queue_duration_s":4.637906,"scheduling_latency_s":4.637928}1762vm-test-run-nixbot-gitlab> gitlab # [  192.196902] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.318Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dadbad6769e0d587f70f318f","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dadbad6769e0d587f70f318f: done: 0.015013 sec","job_status":"done","queue_duration_s":4.637906,"scheduling_latency_s":4.637928,"redis_calls":13,"redis_duration_s":0.003077,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000904,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002173,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009753,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015013,"completed_at":"2026-06-14T15:02:08.318Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1763vm-test-run-nixbot-gitlab> gitlab # [  192.289317] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.319Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9184e817253e2cdc59d7dce","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9184e817253e2cdc59d7dce: start","job_status":"start","queue_duration_s":4.654128,"scheduling_latency_s":4.65415}1764vm-test-run-nixbot-gitlab> gitlab # [  192.290928] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.334Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9184e817253e2cdc59d7dce","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9184e817253e2cdc59d7dce: done: 0.014677 sec","job_status":"done","queue_duration_s":4.654128,"scheduling_latency_s":4.65415,"redis_calls":13,"redis_duration_s":0.002646,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000799,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001847,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009678,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014677,"completed_at":"2026-06-14T15:02:08.333Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1765vm-test-run-nixbot-gitlab> gitlab # [  192.373307] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.335Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d7b4ebe89a6cc1b847c6e46","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d7b4ebe89a6cc1b847c6e46: start","job_status":"start","queue_duration_s":4.670004,"scheduling_latency_s":4.670025}1766vm-test-run-nixbot-gitlab> gitlab # [  192.374903] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.350Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d7b4ebe89a6cc1b847c6e46","created_at":"2026-06-14T15:02:03.603Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-9ca484a1497d4342","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2df781056393bece9e377b8b2aa2f87d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:03.665Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d7b4ebe89a6cc1b847c6e46: done: 0.014956 sec","job_status":"done","queue_duration_s":4.670004,"scheduling_latency_s":4.670025,"redis_calls":13,"redis_duration_s":0.0030050000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000873,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002132,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009905,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014956,"completed_at":"2026-06-14T15:02:08.350Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1767vm-test-run-nixbot-gitlab> gitlab # [  192.465351] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.351Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c06e35f0431ca9635ed8fbf3","created_at":"2026-06-14T15:02:04.291Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-fac18b3bb3c64878","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f3b21432cb86e754d64b3c25bf76cba0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:04.307Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c06e35f0431ca9635ed8fbf3: start","job_status":"start","queue_duration_s":4.043504,"scheduling_latency_s":4.043526}1768vm-test-run-nixbot-gitlab> gitlab # [  192.466119] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.361Z","message":"No existing merge request to be cleaned up."}1769vm-test-run-nixbot-gitlab> gitlab # [  192.467049] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.367Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c06e35f0431ca9635ed8fbf3","created_at":"2026-06-14T15:02:04.291Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-fac18b3bb3c64878","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f3b21432cb86e754d64b3c25bf76cba0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:04.307Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c06e35f0431ca9635ed8fbf3: done: 0.016099 sec","job_status":"done","queue_duration_s":4.043504,"scheduling_latency_s":4.043526,"redis_calls":10,"redis_duration_s":0.002091,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000809,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001282,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010638,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016099,"completed_at":"2026-06-14T15:02:08.367Z","load_balancing_strategy":"primary","db_duration_s":0.001718,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1770vm-test-run-nixbot-gitlab> gitlab # [  192.539843] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.368Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"26e23bf4c8ada8689e1965df","created_at":"2026-06-14T15:02:04.291Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-fac18b3bb3c64878","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f3b21432cb86e754d64b3c25bf76cba0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:04.307Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-26e23bf4c8ada8689e1965df: start","job_status":"start","queue_duration_s":4.060915,"scheduling_latency_s":4.060937}1771vm-test-run-nixbot-gitlab> gitlab # [  192.541774] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.377Z","message":"No existing merge request to be cleaned up."}1772vm-test-run-nixbot-gitlab> gitlab # [  192.542902] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.383Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"26e23bf4c8ada8689e1965df","created_at":"2026-06-14T15:02:04.291Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-fac18b3bb3c64878","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f3b21432cb86e754d64b3c25bf76cba0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:04.307Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-26e23bf4c8ada8689e1965df: done: 0.014922 sec","job_status":"done","queue_duration_s":4.060915,"scheduling_latency_s":4.060937,"redis_calls":10,"redis_duration_s":0.002144,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000821,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001323,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009804,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014922,"completed_at":"2026-06-14T15:02:08.383Z","load_balancing_strategy":"primary","db_duration_s":0.001344,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1773vm-test-run-nixbot-gitlab> gitlab # [  192.614171] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.385Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c8049d759eacd3f5f77cedc5","created_at":"2026-06-14T15:02:04.291Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-fac18b3bb3c64878","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f3b21432cb86e754d64b3c25bf76cba0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:04.307Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c8049d759eacd3f5f77cedc5: start","job_status":"start","queue_duration_s":4.077094,"scheduling_latency_s":4.077115}1774vm-test-run-nixbot-gitlab> gitlab # [  192.616069] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.398Z","message":"No existing merge request to be cleaned up."}1775vm-test-run-nixbot-gitlab> gitlab # [  192.617379] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.404Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c8049d759eacd3f5f77cedc5","created_at":"2026-06-14T15:02:04.291Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-fac18b3bb3c64878","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f3b21432cb86e754d64b3c25bf76cba0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:04.307Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c8049d759eacd3f5f77cedc5: done: 0.019371 sec","job_status":"done","queue_duration_s":4.077094,"scheduling_latency_s":4.077115,"redis_calls":10,"redis_duration_s":0.002361,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000996,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001365,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009663,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019371,"completed_at":"2026-06-14T15:02:08.404Z","load_balancing_strategy":"primary","db_duration_s":0.009046,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1776vm-test-run-nixbot-gitlab> gitlab # [  192.660646] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.405Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"fb69273cc8b330eba43701b9","created_at":"2026-06-14T15:02:04.291Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-fac18b3bb3c64878","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f3b21432cb86e754d64b3c25bf76cba0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:04.307Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-fb69273cc8b330eba43701b9: start","job_status":"start","queue_duration_s":4.097722,"scheduling_latency_s":4.097744}1777vm-test-run-nixbot-gitlab> gitlab # [  192.736503] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.414Z","message":"No existing merge request to be cleaned up."}1778vm-test-run-nixbot-gitlab> gitlab # [  192.737272] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.420Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"fb69273cc8b330eba43701b9","created_at":"2026-06-14T15:02:04.291Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-fac18b3bb3c64878","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f3b21432cb86e754d64b3c25bf76cba0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:04.307Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-fb69273cc8b330eba43701b9: done: 0.014778 sec","job_status":"done","queue_duration_s":4.097722,"scheduling_latency_s":4.097744,"redis_calls":10,"redis_duration_s":0.002071,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000804,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001267,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009734,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014778,"completed_at":"2026-06-14T15:02:08.420Z","load_balancing_strategy":"primary","db_duration_s":0.001365,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1779vm-test-run-nixbot-gitlab> gitlab # [  192.813844] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.421Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Projects::RefreshBuildArtifactsSizeStatisticsWorker","args":[],"jid":"c26606e42c5a3ab35e98d3ce","created_at":"2026-06-14T15:02:04.431Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-c9cd3442454544bd","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","correlation_id":"0e5940dc85f904097da1997d80fd3735","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:04.446Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::RefreshBuildArtifactsSizeStatisticsWorker JID-c26606e42c5a3ab35e98d3ce: start","job_status":"start","queue_duration_s":3.974919,"scheduling_latency_s":3.97494}1780vm-test-run-nixbot-gitlab> gitlab # [  192.814901] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:08.445Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Projects::RefreshBuildArtifactsSizeStatisticsWorker","args":[],"jid":"c26606e42c5a3ab35e98d3ce","created_at":"2026-06-14T15:02:04.431Z","trace_propagation_headers":{"sentry-trace":"50716a60276a43ea98447b818d160177-c9cd3442454544bd","baggage":"sentry-trace_id=50716a60276a43ea98447b818d160177,sentry-release=Unknown"},"meta.caller_id":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","correlation_id":"0e5940dc85f904097da1997d80fd3735","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:04.446Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::RefreshBuildArtifactsSizeStatisticsWorker JID-c26606e42c5a3ab35e98d3ce: done: 0.023881 sec","job_status":"done","queue_duration_s":3.974919,"scheduling_latency_s":3.97494,"redis_calls":13,"redis_duration_s":0.002749,"redis_read_bytes":12,"redis_write_bytes":1665,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000467,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":345,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":519,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00123,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":801,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015245,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000376864,"duration_s":0.023881,"completed_at":"2026-06-14T15:02:08.445Z","load_balancing_strategy":"primary","db_duration_s":0.003062,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1781vm-test-run-nixbot-gitlab> gitlab # [  192.904101] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:09.638Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"03b2e45fc9d69b3aa45053dd","created_at":"2026-06-14T15:02:09.630Z","trace_propagation_headers":{"sentry-trace":"2f32f832f09b4c9db5cf7eefe02186bd-31d85681d344480b","baggage":"sentry-trace_id=2f32f832f09b4c9db5cf7eefe02186bd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV3AF1DN5Y5RCD6J8RHJZRC8","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:09.636Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-03b2e45fc9d69b3aa45053dd: start","job_status":"start","queue_duration_s":0.001281,"scheduling_latency_s":0.001317}1782vm-test-run-nixbot-gitlab> gitlab # [  192.906129] setup-gitlab-start[2943]: [master (root-commit) 5bc9257] initial commit1783vm-test-run-nixbot-gitlab> gitlab # [  192.907363] setup-gitlab-start[2943]:  1 file changed, 10 insertions(+)1784vm-test-run-nixbot-gitlab> gitlab # [  192.908834] setup-gitlab-start[2943]:  create mode 100644 flake.nix1785vm-test-run-nixbot-gitlab> gitlab # [  192.910261] gitaly[1387]: time="2026-06-14T15:02:09.699Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AF1DN5Y5RCD6J8RHJZRC8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-14T15:02:19.213" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T15:02:09.693" grpc.time_ms=5.263 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1786vm-test-run-nixbot-gitlab> gitlab # [  192.911337] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:09.683Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"03b2e45fc9d69b3aa45053dd","created_at":"2026-06-14T15:02:09.630Z","trace_propagation_headers":{"sentry-trace":"2f32f832f09b4c9db5cf7eefe02186bd-31d85681d344480b","baggage":"sentry-trace_id=2f32f832f09b4c9db5cf7eefe02186bd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV3AF1DN5Y5RCD6J8RHJZRC8","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:09.636Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-03b2e45fc9d69b3aa45053dd: done: 0.04538 sec","job_status":"done","queue_duration_s":0.001281,"scheduling_latency_s":0.001317,"redis_calls":10,"redis_duration_s":0.0026100000000000003,"redis_read_bytes":8,"redis_write_bytes":1165,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001544,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001066,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":7,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028686,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.026314518,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.04538,"completed_at":"2026-06-14T15:02:09.683Z","load_balancing_strategy":"primary","db_duration_s":0.008976,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1787vm-test-run-nixbot-gitlab> gitlab # [  192.995204] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:09.685Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"099ca747c2b4980ed03dd390","created_at":"2026-06-14T15:02:09.642Z","trace_propagation_headers":{"sentry-trace":"2f32f832f09b4c9db5cf7eefe02186bd-31d85681d344480b","baggage":"sentry-trace_id=2f32f832f09b4c9db5cf7eefe02186bd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV3AF1DN5Y5RCD6J8RHJZRC8","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:09.649Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-099ca747c2b4980ed03dd390: start","job_status":"start","queue_duration_s":0.035287,"scheduling_latency_s":0.035315}1788vm-test-run-nixbot-gitlab> gitlab # [  192.996870] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:09.759Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"099ca747c2b4980ed03dd390","created_at":"2026-06-14T15:02:09.642Z","trace_propagation_headers":{"sentry-trace":"2f32f832f09b4c9db5cf7eefe02186bd-31d85681d344480b","baggage":"sentry-trace_id=2f32f832f09b4c9db5cf7eefe02186bd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV3AF1DN5Y5RCD6J8RHJZRC8","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:09.649Z","job_size_bytes":2,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-099ca747c2b4980ed03dd390: done: 0.074532 sec","job_status":"done","queue_duration_s":0.035287,"scheduling_latency_s":0.035315,"redis_calls":14,"redis_duration_s":0.0039239999999999995,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000659,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001707,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001558,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.052066,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.053022382,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.074532,"completed_at":"2026-06-14T15:02:09.759Z","load_balancing_strategy":"primary","db_duration_s":0.015248,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1789vm-test-run-nixbot-gitlab> gitlab # [  193.069514] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:09.869Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"0541d4a9dc23e21ae5112a93","created_at":"2026-06-14T15:02:09.862Z","trace_propagation_headers":{"sentry-trace":"2f32f832f09b4c9db5cf7eefe02186bd-31d85681d344480b","baggage":"sentry-trace_id=2f32f832f09b4c9db5cf7eefe02186bd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV3AF1DN5Y5RCD6J8RHJZRC8","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:09.867Z","job_size_bytes":5,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-0541d4a9dc23e21ae5112a93: start","job_status":"start","queue_duration_s":0.001421,"scheduling_latency_s":0.00146}1790vm-test-run-nixbot-gitlab> gitlab # [  193.076520] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:09.951Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"0541d4a9dc23e21ae5112a93","created_at":"2026-06-14T15:02:09.862Z","trace_propagation_headers":{"sentry-trace":"2f32f832f09b4c9db5cf7eefe02186bd-31d85681d344480b","baggage":"sentry-trace_id=2f32f832f09b4c9db5cf7eefe02186bd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV3AF1DN5Y5RCD6J8RHJZRC8","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:09.867Z","job_size_bytes":5,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-0541d4a9dc23e21ae5112a93: done: 0.082449 sec","job_status":"done","queue_duration_s":0.001421,"scheduling_latency_s":0.00146,"redis_calls":12,"redis_duration_s":0.0028510000000000002,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.00069,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001693,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000468,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.058844,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000332445,"duration_s":0.082449,"completed_at":"2026-06-14T15:02:09.951Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016436,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1791vm-test-run-nixbot-gitlab> gitlab # [  193.164073] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:09.953Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"b1e116f2153c01547006cdda","created_at":"2026-06-14T15:02:09.889Z","trace_propagation_headers":{"sentry-trace":"2f32f832f09b4c9db5cf7eefe02186bd-31d85681d344480b","baggage":"sentry-trace_id=2f32f832f09b4c9db5cf7eefe02186bd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV3AF1DN5Y5RCD6J8RHJZRC8","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:09.894Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-b1e116f2153c01547006cdda: start","job_status":"start","queue_duration_s":0.058939,"scheduling_latency_s":0.058961}1792vm-test-run-nixbot-gitlab> gitlab # [  193.176319] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:10.102Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"b1e116f2153c01547006cdda","created_at":"2026-06-14T15:02:09.889Z","trace_propagation_headers":{"sentry-trace":"2f32f832f09b4c9db5cf7eefe02186bd-31d85681d344480b","baggage":"sentry-trace_id=2f32f832f09b4c9db5cf7eefe02186bd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV3AF1DN5Y5RCD6J8RHJZRC8","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:09.894Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-b1e116f2153c01547006cdda: done: 0.149238 sec","job_status":"done","queue_duration_s":0.058939,"scheduling_latency_s":0.058961,"redis_calls":11,"redis_duration_s":0.0024100000000000002,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.00068,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001329,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000401,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.113,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.113,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.099541,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000344178,"duration_s":0.149238,"completed_at":"2026-06-14T15:02:10.102Z","load_balancing_strategy":"primary","db_duration_s":0.050892,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1793vm-test-run-nixbot-gitlab> gitlab # [  193.261314] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:10.103Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"0c29299117aeb91dead6414c","created_at":"2026-06-14T15:02:09.896Z","trace_propagation_headers":{"sentry-trace":"2f32f832f09b4c9db5cf7eefe02186bd-31d85681d344480b","baggage":"sentry-trace_id=2f32f832f09b4c9db5cf7eefe02186bd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV3AF1DN5Y5RCD6J8RHJZRC8","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:09.900Z","job_size_bytes":89,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-0c29299117aeb91dead6414c: start","job_status":"start","queue_duration_s":0.202875,"scheduling_latency_s":0.202896}1794vm-test-run-nixbot-gitlab> gitlab # [  193.342248] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:10.149Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"0c29299117aeb91dead6414c","created_at":"2026-06-14T15:02:09.896Z","trace_propagation_headers":{"sentry-trace":"2f32f832f09b4c9db5cf7eefe02186bd-31d85681d344480b","baggage":"sentry-trace_id=2f32f832f09b4c9db5cf7eefe02186bd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV3AF1DN5Y5RCD6J8RHJZRC8","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:09.900Z","job_size_bytes":89,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-0c29299117aeb91dead6414c: done: 0.046209 sec","job_status":"done","queue_duration_s":0.202875,"scheduling_latency_s":0.202896,"redis_calls":11,"redis_duration_s":0.002535,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000593,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001411,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000531,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025674,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000395861,"duration_s":0.046209,"completed_at":"2026-06-14T15:02:10.149Z","load_balancing_strategy":"primary","db_duration_s":0.015478,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1795vm-test-run-nixbot-gitlab> gitlab # [  193.348545] setup-gitlab-start[2618]: + git remote add origin http://oauth2:glpat-T9cZOZ5lxCGBJ6y2CDCkeG86MQp1OjEH.01.0w0mp5g2p@gitlab/root/test-flake.git1796vm-test-run-nixbot-gitlab> gitlab # [  193.420309] setup-gitlab-start[2618]: + git push -u origin master1797vm-test-run-nixbot-gitlab> gitlab # [  193.421779] gitaly[1387]: time="2026-06-14T15:02:09.721Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KV3AF1DN5Y5RCD6J8RHJZRC8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-14T15:02:19.240" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T15:02:09.720" grpc.time_ms=0.127 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1798vm-test-run-nixbot-gitlab> gitlab # [  193.422916] gitaly[1387]: time="2026-06-14T15:02:09.728Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KV3AF1DN5Y5RCD6J8RHJZRC8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-14T15:02:19.247" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T15:02:09.727" grpc.time_ms=0.198 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1799vm-test-run-nixbot-gitlab> gitlab # [  193.424138] gitaly[1387]: time="2026-06-14T15:02:09.917Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AF1DN5Y5RCD6J8RHJZRC8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-14T15:02:19.001" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T15:02:09.912" grpc.time_ms=5.483 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1800vm-test-run-nixbot-gitlab> gitlab # [  193.425083] gitaly[1387]: time="2026-06-14T15:02:10.614Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV3AF3SEKWAGDVDRVEWKNTJH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T15:02:10.608" grpc.time_ms=5.847 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1801vm-test-run-nixbot-gitlab> gitlab # [  193.432443] gitaly[1387]: time="2026-06-14T15:02:11.918Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=20 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-14T15:02:21.006" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T15:02:11.896" grpc.time_ms=22.024 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1802vm-test-run-nixbot-gitlab> gitlab # [  193.509450] gitaly[1387]: time="2026-06-14T15:02:11.946Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=331 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-06-14T15:02:41.039" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-06-14T15:02:11.939" grpc.time_ms=7.317 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1803vm-test-run-nixbot-gitlab> gitlab # [  193.510765] gitaly[1387]: time="2026-06-14T15:02:11.961Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ duration_ms=255ns method=POST pid=1387 status=200 url="http://unix/api/v4/internal/allowed"1804vm-test-run-nixbot-gitlab> gitlab # [  193.511931] gitaly[1387]: time="2026-06-14T15:02:12.000Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ duration_ms=38ns method=POST pid=1387 status=200 url="http://unix/api/v4/internal/pre_receive"1805vm-test-run-nixbot-gitlab> gitlab # [  193.513223] gitaly[1387]: time="2026-06-14T15:02:12.000Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5322 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T15:02:11.705" grpc.time_ms=294.879 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1806vm-test-run-nixbot-gitlab> gitlab # [  193.514099] gitaly[1387]: time="2026-06-14T15:02:12.023Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4792 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T15:02:12.022" grpc.time_ms=0.453 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1807vm-test-run-nixbot-gitlab> gitlab # [  193.520578] gitaly[1387]: time="2026-06-14T15:02:12.050Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4791 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T15:02:12.049" grpc.time_ms=0.343 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1808vm-test-run-nixbot-gitlab> gitlab # [  193.593257] gitaly[1387]: time="2026-06-14T15:02:12.138Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4793 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T15:02:12.137" grpc.time_ms=0.952 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1809vm-test-run-nixbot-gitlab> gitlab # [  193.671839] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:12.931Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 5bc925784851d57384be462c5a4edebddba61b3c refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"cc0dfbbebe6b2b191b6b9c09","created_at":"2026-06-14T15:02:12.926Z","trace_propagation_headers":{"sentry-trace":"150056be3f72423bb0ba2d82ba490904-bf4b95f01d2e47d1","baggage":"sentry-trace_id=150056be3f72423bb0ba2d82ba490904,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV3AF3Y0Q8499QWKYN7TAAGQ","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:12.930Z","job_size_bytes":185,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-cc0dfbbebe6b2b191b6b9c09: start","job_status":"start","queue_duration_s":0.001435,"scheduling_latency_s":0.00147}1810vm-test-run-nixbot-gitlab> gitlab # [  193.695864] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:12.958Z","message":"changes: \"0000000000000000000000000000000000000000 5bc925784851d57384be462c5a4edebddba61b3c refs/heads/master\\n\""}1811vm-test-run-nixbot-gitlab> gitlab # [  193.710603] gitaly[1387]: time="2026-06-14T15:02:12.972Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=24 command.spawn_token_fork_ms=19 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-14T15:02:22.036" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T15:02:12.946" grpc.time_ms=25.71 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1812vm-test-run-nixbot-gitlab> gitlab # [  193.722783] gitaly[1387]: time="2026-06-14T15:02:12.985Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-14T15:02:22.008" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T15:02:12.978" grpc.time_ms=6.326 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1813vm-test-run-nixbot-gitlab> gitlab # [  193.743573] gitaly[1387]: time="2026-06-14T15:02:13.005Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-14T15:02:22.029" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T15:02:12.999" grpc.time_ms=5.975 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1814vm-test-run-nixbot-gitlab> gitlab # [  193.760528] gitaly[1387]: time="2026-06-14T15:02:13.022Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1563 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-06-14T15:02:23.010" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T15:02:13.010" grpc.time_ms=11.921 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1815vm-test-run-nixbot-gitlab> gitlab # [  193.794351] gitaly[1387]: time="2026-06-14T15:02:13.056Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-06-14T15:02:23.048" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=226 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T15:02:13.048" grpc.time_ms=8.213 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1816vm-test-run-nixbot-gitlab> gitlab # [  193.826181] gitaly[1387]: time="2026-06-14T15:02:13.088Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3942 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T15:02:13.087" grpc.time_ms=1.081 pid=1387 span.kind=server system=grpc1817vm-test-run-nixbot-gitlab> gitlab # [  193.846088] gitaly[1387]: time="2026-06-14T15:02:13.108Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=948 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=5 command.system_time_ms=1 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-06-14T15:02:23.004" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T15:02:13.095" grpc.time_ms=13.15 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1818vm-test-run-nixbot-gitlab> gitlab # [  193.859172] gitaly[1387]: time="2026-06-14T15:02:13.119Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3944 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T15:02:13.118" grpc.time_ms=0.283 pid=1387 span.kind=server system=grpc1819vm-test-run-nixbot-gitlab> gitlab # [  193.885619] gitaly[1387]: time="2026-06-14T15:02:13.121Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ duration_ms=222ns method=POST pid=1387 status=200 url="http://unix/api/v4/internal/post_receive"1820vm-test-run-nixbot-gitlab> gitlab # [  193.887153] gitaly[1387]: time="2026-06-14T15:02:13.149Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4816 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T15:02:12.898" grpc.time_ms=250.962 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1821vm-test-run-nixbot-gitlab> gitlab # [  193.920498] gitaly[1387]: time="2026-06-14T15:02:13.182Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ diskcache=cd5f81ef-d152-4d59-8a67-61f38d13de41 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-14T15:02:23.064" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T15:02:13.064" pid=1387 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1822vm-test-run-nixbot-gitlab> gitlab # [  193.922300] gitaly[1387]: time="2026-06-14T15:02:13.184Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ diskcache=ebf9c15e-6302-4b58-ab16-55117acdce1c grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T15:02:10.711" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1823vm-test-run-nixbot-gitlab> gitlab # [  193.954392] gitaly[1387]: time="2026-06-14T15:02:13.216Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=323 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=901120 command.minflt=13367 command.oublock=32 command.real_time_ms=2459 command.spawn_token_fork_ms=3 command.system_time_ms=241 command.user_time_ms=82 component=gitaly.StreamServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=684 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=58 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T15:02:10.711" grpc.time_ms=2472.601 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1824vm-test-run-nixbot-gitlab> gitlab # [  193.976523] gitaly[1387]: time="2026-06-14T15:02:13.216Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=63 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4897 command.oublock=8 command.real_time_ms=107 command.spawn_token_fork_ms=1 command.system_time_ms=44 command.user_time_ms=19 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-14T15:02:23.064" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T15:02:13.064" grpc.time_ms=118.451 pid=1387 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1825vm-test-run-nixbot-gitlab> gitlab # [  193.977492] setup-gitlab-start[2947]: To http://gitlab/root/test-flake.git1826vm-test-run-nixbot-gitlab> gitlab # [  193.978760] setup-gitlab-start[2947]:  * [new branch]      master -> master1827vm-test-run-nixbot-gitlab> gitlab # [  193.984184] setup-gitlab-start[2947]: branch 'master' set up to track 'origin/master'.1828vm-test-run-nixbot-gitlab> gitlab # [  193.985352] gitlab-workhorse[1290]: time="2026-06-14T15:02:13Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521829vm-test-run-nixbot-gitlab> gitlab # [  193.986198] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:13 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 25921830vm-test-run-nixbot-gitlab> gitlab # [  193.987151] gitaly[1387]: time="2026-06-14T15:02:13.234Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-14T15:02:23.233" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T15:02:13.233" grpc.time_ms=0.174 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1831vm-test-run-nixbot-gitlab> gitlab # [  193.990113] systemd[1]: Finished setup-gitlab.service.1832vm-test-run-nixbot-gitlab> gitlab # [  193.991127] systemd[1]: Starting nixbot CI service...1833vm-test-run-nixbot-gitlab> gitlab # [  193.992478] gitaly[1387]: time="2026-06-14T15:02:13.253Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=934 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-06-14T15:02:23.237" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T15:02:13.237" grpc.time_ms=15.582 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1834vm-test-run-nixbot-gitlab> gitlab # [  194.045822] gitaly[1387]: time="2026-06-14T15:02:13.308Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=341 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-06-14T15:02:43.003" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:13.303" grpc.time_ms=4.121 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1835vm-test-run-nixbot-gitlab> gitlab # [  194.066908] gitaly[1387]: time="2026-06-14T15:02:13.329Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-14T15:02:23.008" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T15:02:13.328" grpc.time_ms=0.22 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1836vm-test-run-nixbot-gitlab> gitlab # [  194.073484] gitaly[1387]: time="2026-06-14T15:02:13.335Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=344 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-06-14T15:02:23.011" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-06-14T15:02:13.331" grpc.time_ms=3.824 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1837vm-test-run-nixbot-gitlab> gitlab # [  194.090908] gitaly[1387]: time="2026-06-14T15:02:13.353Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-14T15:02:23.027" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T15:02:13.347" grpc.time_ms=5.246 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1838vm-test-run-nixbot-gitlab> gitlab # [  194.100399] gitaly[1387]: time="2026-06-14T15:02:13.362Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-14T15:02:23.037" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T15:02:13.357" grpc.time_ms=5.091 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1839vm-test-run-nixbot-gitlab> gitlab # [  194.116949] gitaly[1387]: time="2026-06-14T15:02:13.379Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=963 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-14T15:02:43.070" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:13.370" grpc.time_ms=8.509 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1840vm-test-run-nixbot-gitlab> gitlab # [  194.412805] gitaly[1387]: time="2026-06-14T15:02:13.674Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-06-14T15:02:23.003" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:13.673" grpc.time_ms=0.806 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1841vm-test-run-nixbot-gitlab> gitlab # [  194.467076] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:13.705Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 5bc925784851d57384be462c5a4edebddba61b3c refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"cc0dfbbebe6b2b191b6b9c09","created_at":"2026-06-14T15:02:12.926Z","trace_propagation_headers":{"sentry-trace":"150056be3f72423bb0ba2d82ba490904-bf4b95f01d2e47d1","baggage":"sentry-trace_id=150056be3f72423bb0ba2d82ba490904,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV3AF3Y0Q8499QWKYN7TAAGQ","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:12.930Z","job_size_bytes":185,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-cc0dfbbebe6b2b191b6b9c09: done: 0.773507 sec","job_status":"done","queue_duration_s":0.001435,"scheduling_latency_s":0.00147,"gitaly_calls":14,"gitaly_duration_s":0.270199,"redis_calls":212,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.048778,"redis_read_bytes":1568,"redis_write_bytes":21034,"redis_cache_calls":14,"redis_cache_duration_s":0.003787,"redis_cache_read_bytes":16,"redis_cache_write_bytes":884,"redis_feature_flag_calls":43,"redis_feature_flag_duration_s":0.011482,"redis_feature_flag_read_bytes":1186,"redis_feature_flag_write_bytes":5432,"redis_queues_calls":8,"redis_queues_duration_s":0.001245,"redis_queues_read_bytes":8,"redis_queues_write_bytes":4932,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002146,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1259,"redis_repository_cache_calls":92,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.01915,"redis_repository_cache_read_bytes":214,"redis_repository_cache_write_bytes":6392,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.010968,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":54,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":54,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.036,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":54,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.227,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.241,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.036,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.413297,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000571022,"exclusive_lock_hold_duration_s":0.020719114,"duration_s":0.773507,"completed_at":"2026-06-14T15:02:13.705Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.059206,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1842vm-test-run-nixbot-gitlab> gitlab # [  194.482701] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:13.706Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"e794b8035721a523d2ff0c2d","created_at":"2026-06-14T15:02:13.024Z","trace_propagation_headers":{"sentry-trace":"150056be3f72423bb0ba2d82ba490904-afee005142d54d83","baggage":"sentry-trace_id=150056be3f72423bb0ba2d82ba490904,sentry-release=Unknown"},"correlation_id":"01KV3AF3Y0Q8499QWKYN7TAAGQ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:13.029Z","job_size_bytes":46,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-e794b8035721a523d2ff0c2d: start","job_status":"start","queue_duration_s":0.677586,"scheduling_latency_s":0.677608}1843vm-test-run-nixbot-gitlab> gitlab # [  194.607541] gitaly[1387]: time="2026-06-14T15:02:13.869Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-06-14T21:02:13.865" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T15:02:13.866" grpc.time_ms=2.593 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1844vm-test-run-nixbot-gitlab> gitlab # [  194.624561] gitaly[1387]: time="2026-06-14T15:02:13.886Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=649 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-06-14T15:02:43.576" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:13.876" grpc.time_ms=9.647 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1845vm-test-run-nixbot-gitlab> gitlab # [  194.651510] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:13.913Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"e794b8035721a523d2ff0c2d","created_at":"2026-06-14T15:02:13.024Z","trace_propagation_headers":{"sentry-trace":"150056be3f72423bb0ba2d82ba490904-afee005142d54d83","baggage":"sentry-trace_id=150056be3f72423bb0ba2d82ba490904,sentry-release=Unknown"},"correlation_id":"01KV3AF3Y0Q8499QWKYN7TAAGQ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:13.029Z","job_size_bytes":46,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-e794b8035721a523d2ff0c2d: done: 0.206842 sec","job_status":"done","queue_duration_s":0.677586,"scheduling_latency_s":0.677608,"gitaly_calls":2,"gitaly_duration_s":0.017699,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.009751,"redis_read_bytes":68,"redis_write_bytes":4886,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001927,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000897,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2115,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002363,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.003772,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":752,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000792,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.131,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.131,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.046103,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000573816,"duration_s":0.206842,"completed_at":"2026-06-14T15:02:13.913Z","load_balancing_strategy":"primary","db_duration_s":0.134536,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1846vm-test-run-nixbot-gitlab> gitlab # [  194.657486] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:13.915Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"bae532350673a50dc2a5449d","created_at":"2026-06-14T15:02:13.041Z","trace_propagation_headers":{"sentry-trace":"150056be3f72423bb0ba2d82ba490904-afee005142d54d83","baggage":"sentry-trace_id=150056be3f72423bb0ba2d82ba490904,sentry-release=Unknown"},"correlation_id":"01KV3AF3Y0Q8499QWKYN7TAAGQ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:13.044Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-bae532350673a50dc2a5449d: start","job_status":"start","queue_duration_s":0.8706,"scheduling_latency_s":0.8707}1847vm-test-run-nixbot-gitlab> gitlab # [  194.732094] gitaly[1387]: time="2026-06-14T15:02:13.963Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ diskcache=3ba8941d-e41d-44c3-9c32-fa3659225657 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-14T21:02:13.938" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:13.939" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1848vm-test-run-nixbot-gitlab> gitlab # [  194.733354] gitaly[1387]: time="2026-06-14T15:02:13.964Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1287 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-14T21:02:13.938" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:13.939" grpc.time_ms=24.735 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1849vm-test-run-nixbot-gitlab> gitlab # [  194.737867] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:13.999Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"bae532350673a50dc2a5449d","created_at":"2026-06-14T15:02:13.041Z","trace_propagation_headers":{"sentry-trace":"150056be3f72423bb0ba2d82ba490904-afee005142d54d83","baggage":"sentry-trace_id=150056be3f72423bb0ba2d82ba490904,sentry-release=Unknown"},"correlation_id":"01KV3AF3Y0Q8499QWKYN7TAAGQ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:13.044Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-bae532350673a50dc2a5449d: done: 0.084491 sec","job_status":"done","queue_duration_s":0.8706,"scheduling_latency_s":0.8707,"gitaly_calls":1,"gitaly_duration_s":0.027217,"redis_calls":15,"redis_duration_s":0.006393,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00086,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002423,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001083,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002027,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.009,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.018,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030681,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.060506292999999996,"duration_s":0.084491,"completed_at":"2026-06-14T15:02:13.999Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.019457,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1850vm-test-run-nixbot-gitlab> gitlab # [  194.818590] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:14.002Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"4fcd85cc4e7c7c8657be59a7","created_at":"2026-06-14T15:02:13.229Z","trace_propagation_headers":{"sentry-trace":"150056be3f72423bb0ba2d82ba490904-afee005142d54d83","baggage":"sentry-trace_id=150056be3f72423bb0ba2d82ba490904,sentry-release=Unknown"},"correlation_id":"01KV3AF3Y0Q8499QWKYN7TAAGQ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:13.231Z","job_size_bytes":23,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-4fcd85cc4e7c7c8657be59a7: start","job_status":"start","queue_duration_s":0.770379,"scheduling_latency_s":0.770401}1851vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for unit setup-gitlab.service, in 195.86 seconds)1852vm-test-run-nixbot-gitlab> gitlab: waiting for unit nixbot.service1853vm-test-run-nixbot-gitlab> gitlab # [  194.825789] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:14.043Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"4fcd85cc4e7c7c8657be59a7","created_at":"2026-06-14T15:02:13.229Z","trace_propagation_headers":{"sentry-trace":"150056be3f72423bb0ba2d82ba490904-afee005142d54d83","baggage":"sentry-trace_id=150056be3f72423bb0ba2d82ba490904,sentry-release=Unknown"},"correlation_id":"01KV3AF3Y0Q8499QWKYN7TAAGQ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:13.231Z","job_size_bytes":23,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-4fcd85cc4e7c7c8657be59a7: done: 0.041706 sec","job_status":"done","queue_duration_s":0.770379,"scheduling_latency_s":0.770401,"gitaly_calls":1,"gitaly_duration_s":0.009036,"redis_calls":16,"redis_duration_s":0.006859,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.000504,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002026,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.00294,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001389,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021063,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.041706,"completed_at":"2026-06-14T15:02:14.043Z","load_balancing_strategy":"primary","db_duration_s":0.003866,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1854vm-test-run-nixbot-gitlab> gitlab # [  194.964116] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:14.045Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"36d808126c889a5109b05ed8","created_at":"2026-06-14T15:02:13.642Z","trace_propagation_headers":{"sentry-trace":"150056be3f72423bb0ba2d82ba490904-afee005142d54d83","baggage":"sentry-trace_id=150056be3f72423bb0ba2d82ba490904,sentry-release=Unknown"},"correlation_id":"01KV3AF3Y0Q8499QWKYN7TAAGQ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0391a57c8019c54f6b8e2688afc9e2b525665b67f5b2f363f6505afeb9eeb2ea","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:13.648Z","job_size_bytes":467,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-36d808126c889a5109b05ed8: start","job_status":"start","queue_duration_s":0.397488,"scheduling_latency_s":0.397511}1855vm-test-run-nixbot-gitlab> gitlab # [  194.965428] gitaly[1387]: time="2026-06-14T15:02:14.035Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=648 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-06-14T15:02:44.027" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:14.028" grpc.time_ms=6.619 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1856vm-test-run-nixbot-gitlab> gitlab # [  194.966305] gitaly[1387]: time="2026-06-14T15:02:14.081Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T15:02:44.080" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:14.081" grpc.time_ms=0.514 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1857vm-test-run-nixbot-gitlab> gitlab # [  194.966811] gitaly[1387]: time="2026-06-14T15:02:14.091Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=2 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-14T15:02:24.004" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T15:02:14.084" grpc.time_ms=5.806 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1858vm-test-run-nixbot-gitlab> gitlab # [  195.046745] gitaly[1387]: time="2026-06-14T15:02:14.180Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-14T15:02:44.179" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:14.179" grpc.time_ms=0.93 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1859vm-test-run-nixbot-gitlab> gitlab # [  195.048329] gitaly[1387]: time="2026-06-14T15:02:14.186Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=20 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-06-14T15:02:44.183" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:14.183" grpc.time_ms=2.287 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1860vm-test-run-nixbot-gitlab> gitlab # [  195.049228] gitaly[1387]: time="2026-06-14T15:02:14.269Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-14T15:02:24.182" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T15:02:14.263" grpc.time_ms=6.458 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1861vm-test-run-nixbot-gitlab> gitlab # [  195.535522] gitaly[1387]: time="2026-06-14T15:02:14.797Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=963 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-14T15:02:44.777" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:14.778" grpc.time_ms=18.659 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1862vm-test-run-nixbot-gitlab> gitlab # [  195.553988] gitaly[1387]: time="2026-06-14T15:02:14.816Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KV3AF3Y0Q8499QWKYN7TAAGQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-06-14T15:02:44.802" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:14.802" grpc.time_ms=13.833 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1863vm-test-run-nixbot-gitlab> gitlab # [  195.589185] sidekiq-cluster[2097]: {"severity":"WARN","time":"2026-06-14T15:02:14.851Z","class":"CreatePipelineWorker","correlation_id":"01KV3AF3Y0Q8499QWKYN7TAAGQ","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"5bc925784851d57384be462c5a4edebddba61b3c","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"5bc925784851d57384be462c5a4edebddba61b3c","ref":"refs/heads/master"},"retry":0}1864vm-test-run-nixbot-gitlab> gitlab # [  195.594554] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:14.856Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"36d808126c889a5109b05ed8","created_at":"2026-06-14T15:02:13.642Z","trace_propagation_headers":{"sentry-trace":"150056be3f72423bb0ba2d82ba490904-afee005142d54d83","baggage":"sentry-trace_id=150056be3f72423bb0ba2d82ba490904,sentry-release=Unknown"},"correlation_id":"01KV3AF3Y0Q8499QWKYN7TAAGQ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0391a57c8019c54f6b8e2688afc9e2b525665b67f5b2f363f6505afeb9eeb2ea","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:13.648Z","job_size_bytes":467,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-36d808126c889a5109b05ed8: done: 0.811056 sec","job_status":"done","queue_duration_s":0.397488,"scheduling_latency_s":0.397511,"gitaly_calls":7,"gitaly_duration_s":0.068596,"redis_calls":53,"redis_duration_s":0.015489,"redis_read_bytes":171,"redis_write_bytes":4427,"redis_cache_calls":4,"redis_cache_duration_s":0.001406,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.007077,"redis_feature_flag_read_bytes":18,"redis_feature_flag_write_bytes":2433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002067,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003886,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001053,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":43,"db_write_count":1,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":43,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.106,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":38,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.101,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.582627,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000340825,"duration_s":0.811056,"completed_at":"2026-06-14T15:02:14.856Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.138917,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1865vm-test-run-nixbot-gitlab> gitlab # [  195.669262] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:14.858Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"93f40c0263cfb80a24d03a37","created_at":"2026-06-14T15:02:13.897Z","trace_propagation_headers":{"sentry-trace":"150056be3f72423bb0ba2d82ba490904-918d283903044eb8","baggage":"sentry-trace_id=150056be3f72423bb0ba2d82ba490904,sentry-release=Unknown"},"correlation_id":"01KV3AF3Y0Q8499QWKYN7TAAGQ","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:13.902Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-93f40c0263cfb80a24d03a37: start","job_status":"start","queue_duration_s":0.955933,"scheduling_latency_s":0.955975}1866vm-test-run-nixbot-gitlab> gitlab # [  195.670512] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:14.895Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"93f40c0263cfb80a24d03a37","created_at":"2026-06-14T15:02:13.897Z","trace_propagation_headers":{"sentry-trace":"150056be3f72423bb0ba2d82ba490904-918d283903044eb8","baggage":"sentry-trace_id=150056be3f72423bb0ba2d82ba490904,sentry-release=Unknown"},"correlation_id":"01KV3AF3Y0Q8499QWKYN7TAAGQ","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:13.902Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-93f40c0263cfb80a24d03a37: done: 0.036861 sec","job_status":"done","queue_duration_s":0.955933,"scheduling_latency_s":0.955975,"redis_calls":19,"redis_duration_s":0.004033,"redis_read_bytes":21,"redis_write_bytes":4287,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000835,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000507,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2258,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001986,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000705,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02416,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.006130109,"duration_s":0.036861,"completed_at":"2026-06-14T15:02:14.895Z","load_balancing_strategy":"primary","db_duration_s":0.005418,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1867vm-test-run-nixbot-gitlab> gitlab # [  195.746312] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:14.897Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"47202140ad84a8abeeeec4d8","created_at":"2026-06-14T15:02:14.884Z","trace_propagation_headers":{"sentry-trace":"150056be3f72423bb0ba2d82ba490904-69ef6215354a4f05","baggage":"sentry-trace_id=150056be3f72423bb0ba2d82ba490904,sentry-release=Unknown"},"correlation_id":"01KV3AF3Y0Q8499QWKYN7TAAGQ","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:14.888Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-47202140ad84a8abeeeec4d8: start","job_status":"start","queue_duration_s":0.0084,"scheduling_latency_s":0.008421}1868vm-test-run-nixbot-gitlab> gitlab # [  195.823971] nixbot[3073]: {"ts": "2026-06-14T15:02:14.888Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1869vm-test-run-nixbot-gitlab> gitlab # [  195.825262] nixbot[3073]: {"ts": "2026-06-14T15:02:14.907Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1870vm-test-run-nixbot-gitlab> gitlab # [  195.826134] nixbot[3073]: {"ts": "2026-06-14T15:02:14.922Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1871vm-test-run-nixbot-gitlab> gitlab # [  195.826916] nixbot[3073]: {"ts": "2026-06-14T15:02:14.925Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1872vm-test-run-nixbot-gitlab> gitlab # [  195.828090] nixbot[3073]: {"ts": "2026-06-14T15:02:14.929Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1873vm-test-run-nixbot-gitlab> gitlab # [  195.828986] nixbot[3073]: {"ts": "2026-06-14T15:02:14.937Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1874vm-test-run-nixbot-gitlab> gitlab # [  195.830277] nixbot[3073]: {"ts": "2026-06-14T15:02:14.940Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1875vm-test-run-nixbot-gitlab> gitlab # [  195.831312] nixbot[3073]: {"ts": "2026-06-14T15:02:14.944Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1876vm-test-run-nixbot-gitlab> gitlab # [  195.832198] nixbot[3073]: {"ts": "2026-06-14T15:02:14.949Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1877vm-test-run-nixbot-gitlab> gitlab # [  195.833086] nixbot[3073]: {"ts": "2026-06-14T15:02:14.951Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1878vm-test-run-nixbot-gitlab> gitlab # [  195.833932] nixbot[3073]: {"ts": "2026-06-14T15:02:14.955Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1879vm-test-run-nixbot-gitlab> gitlab # [  195.835110] nixbot[3073]: {"ts": "2026-06-14T15:02:14.960Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1880vm-test-run-nixbot-gitlab> gitlab # [  195.836058] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:14.971Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"47202140ad84a8abeeeec4d8","created_at":"2026-06-14T15:02:14.884Z","trace_propagation_headers":{"sentry-trace":"150056be3f72423bb0ba2d82ba490904-69ef6215354a4f05","baggage":"sentry-trace_id=150056be3f72423bb0ba2d82ba490904,sentry-release=Unknown"},"correlation_id":"01KV3AF3Y0Q8499QWKYN7TAAGQ","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:14.888Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-47202140ad84a8abeeeec4d8: done: 0.074467 sec","job_status":"done","queue_duration_s":0.0084,"scheduling_latency_s":0.008421,"redis_calls":11,"redis_duration_s":0.00632,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001863,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00403,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000427,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.008,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042462,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000330769,"duration_s":0.074467,"completed_at":"2026-06-14T15:02:14.971Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024016,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1881vm-test-run-nixbot-gitlab> gitlab # [  195.915429] nixbot[3073]: {"ts": "2026-06-14T15:02:14.963Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1882vm-test-run-nixbot-gitlab> gitlab # [  195.916928] nixbot[3073]: {"ts": "2026-06-14T15:02:14.965Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1883vm-test-run-nixbot-gitlab> gitlab # [  195.918200] nixbot[3073]: {"ts": "2026-06-14T15:02:14.969Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1884vm-test-run-nixbot-gitlab> gitlab # [  195.918973] nixbot[3073]: {"ts": "2026-06-14T15:02:14.971Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1885vm-test-run-nixbot-gitlab> gitlab # [  195.920150] nixbot[3073]: {"ts": "2026-06-14T15:02:14.973Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1886vm-test-run-nixbot-gitlab> gitlab # [  195.921047] nixbot[3073]: {"ts": "2026-06-14T15:02:14.975Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1887vm-test-run-nixbot-gitlab> gitlab # [  196.159411] nixbot[3073]: INFO:     Started server process [3073]1888vm-test-run-nixbot-gitlab> gitlab # [  196.160874] nixbot[3073]: INFO:     Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)1889vm-test-run-nixbot-gitlab> gitlab # [  196.165881] nixbot[3073]: INFO:     ::ffff:127.0.0.1:48632 - "GET /health HTTP/1.1" 200 OK1890vm-test-run-nixbot-gitlab> gitlab # [  196.177510] systemd[1]: Started nixbot CI service.1891vm-test-run-nixbot-gitlab> gitlab # [  196.181545] systemd[1]: Reached target Multi-User System.1892vm-test-run-nixbot-gitlab> gitlab # [  196.185077] systemd[1]: Startup finished in 1.530s (kernel) + 4.139s (initrd) + 3min 10.514s (userspace) = 3min 16.184s.1893vm-test-run-nixbot-gitlab> gitlab # [  196.580355] gitaly[1387]: time="2026-06-14T15:02:15.841Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=968 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KV3AF8M1GA5Z8NP66A1WS7ER grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-14T15:02:45.019" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:15.819" grpc.time_ms=21.635 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1894vm-test-run-nixbot-gitlab> gitlab # [  196.977504] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:16 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 8121895vm-test-run-nixbot-gitlab> gitlab # [  196.979492] nixbot[3073]: {"ts": "2026-06-14T15:02:16.239Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1896vm-test-run-nixbot-gitlab> gitlab # [  197.004732] nixbot[3073]: {"ts": "2026-06-14T15:02:16.266Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1897vm-test-run-nixbot-gitlab> gitlab # [  197.081304] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:16 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 651898vm-test-run-nixbot-gitlab> gitlab # [  197.090678] nixbot[3073]: {"ts": "2026-06-14T15:02:16.344Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1899vm-test-run-nixbot-gitlab> gitlab # [  197.100541] nixbot[3073]: {"ts": "2026-06-14T15:02:16.347Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1900vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for unit nixbot.service, in 2.11 seconds)1901vm-test-run-nixbot-gitlab> gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1902vm-test-run-nixbot-gitlab> gitlab # [  197.193612] nixbot[3073]: INFO:     ::ffff:127.0.0.1:48642 - "GET /health HTTP/1.1" 200 OK1903vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.04 seconds)1904vm-test-run-nixbot-gitlab> subtest: project discovered and webhook registered1905vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1906vm-test-run-nixbot-gitlab> gitlab # [  197.217774] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:16 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 868 "" "python-httpx/0.28.1" 1271907vm-test-run-nixbot-gitlab> gitlab # [  197.226897] nixbot[3073]: {"ts": "2026-06-14T15:02:16.480Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""}1908vm-test-run-nixbot-gitlab> gitlab # [  197.280138] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:16 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "curl/8.20.0" 491909vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.09 seconds)1910vm-test-run-nixbot-gitlab> [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-06-14T15:02:16.460Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': False, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]1911vm-test-run-nixbot-gitlab> (finished: subtest: project discovered and webhook registered, in 0.09 seconds)1912vm-test-run-nixbot-gitlab> subtest: push delivers the webhook1913vm-test-run-nixbot-gitlab> gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1914vm-test-run-nixbot-gitlab> gitlab # [  197.358638] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:16 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 261915vm-test-run-nixbot-gitlab> gitlab # [  197.433733] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:16 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 2071916vm-test-run-nixbot-gitlab> gitlab # [  197.436800] nixbot[3073]: {"ts": "2026-06-14T15:02:16.697Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1917vm-test-run-nixbot-gitlab> gitlab # [  197.546183] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:16 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "python-httpx/0.28.1" 981918vm-test-run-nixbot-gitlab> gitlab # [  197.558184] nixbot[3073]: {"ts": "2026-06-14T15:02:16.818Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1919vm-test-run-nixbot-gitlab> gitlab # [  197.596073] gitaly[1387]: time="2026-06-14T15:02:16.857Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=5 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KV3AF9SDZHFQZMYJGQJN63EX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T15:02:16.843" grpc.time_ms=13.676 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1920vm-test-run-nixbot-gitlab> gitlab # [  197.631873] nixbot[3073]: {"ts": "2026-06-14T15:02:16.892Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""}1921vm-test-run-nixbot-gitlab> gitlab # [  197.633278] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:16 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 868 "" "python-httpx/0.28.1" 631922vm-test-run-nixbot-gitlab> gitlab # [  197.634639] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:16 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 202 "" "git/2.54.0" 2741923vm-test-run-nixbot-gitlab> gitlab # Enumerating objects: 5, done.1924vm-test-run-nixbot-gitlab> gitlab # Counting objects:  20% (1/5)Counting objects:  40% (2/5)Counting objects:  60% (3/5)Counting objects:  80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1925vm-test-run-nixbot-gitlab> gitlab # Delta compression using up to 4 threads1926vm-test-run-nixbot-gitlab> gitlab # Compressing objects:  50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1927vm-test-run-nixbot-gitlab> gitlab # Writing objects:  33% (1/3)Writing objects:  66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 274 bytes | 274.00 KiB/s, done.1928vm-test-run-nixbot-gitlab> gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1929vm-test-run-nixbot-gitlab> gitlab # [  197.682266] gitaly[1387]: time="2026-06-14T15:02:16.944Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFA2482HGB1X5XJ9CS1QP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-06-14T15:02:46.134" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T15:02:16.936" grpc.time_ms=8.275 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1930vm-test-run-nixbot-gitlab> gitlab # [  197.692107] gitaly[1387]: time="2026-06-14T15:02:16.953Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFA2482HGB1X5XJ9CS1QP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T15:02:46.152" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:16.952" grpc.time_ms=0.422 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1931vm-test-run-nixbot-gitlab> gitlab # [  197.739357] gitaly[1387]: time="2026-06-14T15:02:17.001Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFA2482HGB1X5XJ9CS1QP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-06-14T15:02:26.004" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:16.994" grpc.time_ms=6.706 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1932vm-test-run-nixbot-gitlab> gitlab # [  197.813130] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:17 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 731 "" "python-httpx/0.28.1" 1731933vm-test-run-nixbot-gitlab> gitlab # [  197.813756] nixbot[3073]: {"ts": "2026-06-14T15:02:17.075Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""}1934vm-test-run-nixbot-gitlab> gitlab # [  197.833821] gitaly[1387]: time="2026-06-14T15:02:17.096Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFA2V7XJP22BVM223BSZZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-06-14T15:02:27.008" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=253 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:17.088" grpc.time_ms=7.214 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1935vm-test-run-nixbot-gitlab> gitlab # [  197.885112] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:17 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 681936vm-test-run-nixbot-gitlab> gitlab # [  197.887095] nixbot[3073]: {"ts": "2026-06-14T15:02:17.148Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""}1937vm-test-run-nixbot-gitlab> gitlab # [  197.889470] nixbot[3073]: {"ts": "2026-06-14T15:02:17.151Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1938vm-test-run-nixbot-gitlab> gitlab # [  197.890366] gitaly[1387]: time="2026-06-14T15:02:17.152Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=331 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV3AFA2V7XJP22BVM223BSZZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-06-14T15:02:47.044" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-06-14T15:02:17.144" grpc.time_ms=6.999 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1939vm-test-run-nixbot-gitlab> gitlab # [  197.901188] gitaly[1387]: time="2026-06-14T15:02:17.163Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KV3AFA2V7XJP22BVM223BSZZ duration_ms=119ns method=POST pid=1387 status=200 url="http://unix/api/v4/internal/allowed"1940vm-test-run-nixbot-gitlab> gitlab # [  197.937486] gitaly[1387]: time="2026-06-14T15:02:17.199Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KV3AFA2V7XJP22BVM223BSZZ duration_ms=34ns method=POST pid=1387 status=200 url="http://unix/api/v4/internal/pre_receive"1941vm-test-run-nixbot-gitlab> gitlab # [  197.940795] gitaly[1387]: time="2026-06-14T15:02:17.199Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV3AFA2V7XJP22BVM223BSZZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5322 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T15:02:17.043" grpc.time_ms=156.113 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1942vm-test-run-nixbot-gitlab> gitlab # [  197.986530] gitaly[1387]: time="2026-06-14T15:02:17.248Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=5 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV3AFAB563TYVK16CAGSBAG6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T15:02:17.239" grpc.time_ms=8.418 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc1943vm-test-run-nixbot-gitlab> gitlab # [  197.988342] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:17 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 601944vm-test-run-nixbot-gitlab> gitlab # [  198.033569] gitaly[1387]: time="2026-06-14T15:02:17.295Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV3AFA2V7XJP22BVM223BSZZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4792 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T15:02:17.294" grpc.time_ms=1.055 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1945vm-test-run-nixbot-gitlab> gitlab # [  198.099542] gitaly[1387]: time="2026-06-14T15:02:17.361Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV3AFA2V7XJP22BVM223BSZZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4791 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T15:02:17.360" grpc.time_ms=1.604 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1946vm-test-run-nixbot-gitlab> gitlab # [  198.110715] gitaly[1387]: time="2026-06-14T15:02:17.372Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFAD4XJDG0GEZY0EYFEF8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T15:02:17.358" pid=1387 remote_ip="2001:db8:1::1" request_sha=083c43590fffa527704b32b8836424e8ed055017 response_bytes=149 span.kind=server system=grpc1947vm-test-run-nixbot-gitlab> gitlab # [  198.114872] gitaly[1387]: time="2026-06-14T15:02:17.372Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFAD4XJDG0GEZY0EYFEF8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T15:02:17.358" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc1948vm-test-run-nixbot-gitlab> gitlab # [  198.120206] gitlab-workhorse[1290]: time="2026-06-14T15:02:17Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1491949vm-test-run-nixbot-gitlab> gitlab # [  198.120990] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:17 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 149 "" "git/2.54.0" 1151950vm-test-run-nixbot-gitlab> gitlab # [  198.121917] gitaly[1387]: time="2026-06-14T15:02:17.373Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFAD4XJDG0GEZY0EYFEF8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T15:02:17.358" grpc.time_ms=14.629 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc1951vm-test-run-nixbot-gitlab> gitlab # [  198.232070] gitaly[1387]: time="2026-06-14T15:02:17.490Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV3AFA2V7XJP22BVM223BSZZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4793 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T15:02:17.490" grpc.time_ms=0.44 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1952vm-test-run-nixbot-gitlab> gitlab # [  198.279187] gitaly[1387]: time="2026-06-14T15:02:17.540Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFAH3KW4FBCNQ2NCWABK0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T15:02:17.515" pid=1387 remote_ip="2001:db8:1::1" request_sha=30795a37e140afa4228afa29f2b87069c08dcad9 response_bytes=373 span.kind=server system=grpc1953vm-test-run-nixbot-gitlab> gitlab # [  198.280161] gitaly[1387]: time="2026-06-14T15:02:17.540Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1231 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFAH3KW4FBCNQ2NCWABK0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T15:02:17.515" negotiation.command=fetch negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc1954vm-test-run-nixbot-gitlab> gitlab # [  198.280973] gitlab-workhorse[1290]: time="2026-06-14T15:02:17Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3731955vm-test-run-nixbot-gitlab> gitlab # [  198.282169] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:17 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 373 "" "git/2.54.0" 1611956vm-test-run-nixbot-gitlab> gitlab # [  198.282996] gitaly[1387]: time="2026-06-14T15:02:17.541Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1231 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFAH3KW4FBCNQ2NCWABK0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=9 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T15:02:17.515" grpc.time_ms=25.617 negotiation.command=fetch negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc1957vm-test-run-nixbot-gitlab> gitlab # [  198.291543] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:17.552Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","5bc925784851d57384be462c5a4edebddba61b3c 2ee4ce11810ac90f03a8181f7104629d7f1fa32e refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"8362e7a59589b81eeaa6819b","created_at":"2026-06-14T15:02:17.548Z","trace_propagation_headers":{"sentry-trace":"6f84511efa8f4b92b9b2f3461e9a0170-23fe25ea91474a23","baggage":"sentry-trace_id=6f84511efa8f4b92b9b2f3461e9a0170,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV3AFA2V7XJP22BVM223BSZZ","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:17.550Z","job_size_bytes":185,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-8362e7a59589b81eeaa6819b: start","job_status":"start","queue_duration_s":0.002096,"scheduling_latency_s":0.00217}1958vm-test-run-nixbot-gitlab> gitlab # [  198.363319] gitaly[1387]: time="2026-06-14T15:02:17.565Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KV3AFA2V7XJP22BVM223BSZZ duration_ms=45ns method=POST pid=1387 status=200 url="http://unix/api/v4/internal/post_receive"1959vm-test-run-nixbot-gitlab> gitlab # [  198.364739] gitaly[1387]: time="2026-06-14T15:02:17.566Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV3AFA2V7XJP22BVM223BSZZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4816 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T15:02:17.520" grpc.time_ms=45.77 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1960vm-test-run-nixbot-gitlab> gitlab # [  198.365974] gitaly[1387]: time="2026-06-14T15:02:17.578Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KV3AFA2V7XJP22BVM223BSZZ diskcache=f09deba9-ef33-42b6-8db4-dbea3004f957 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T15:02:16.985" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1961vm-test-run-nixbot-gitlab> gitlab # [  198.367207] gitlab-workhorse[1290]: time="2026-06-14T15:02:17Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521962vm-test-run-nixbot-gitlab> gitlab # [  198.367994] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:17 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 6551963vm-test-run-nixbot-gitlab> gitlab # [  198.368935] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:17.566Z","message":"changes: \"5bc925784851d57384be462c5a4edebddba61b3c 2ee4ce11810ac90f03a8181f7104629d7f1fa32e refs/heads/master\\n\""}1964vm-test-run-nixbot-gitlab> gitlab # [  198.370155] gitaly[1387]: time="2026-06-14T15:02:17.581Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=226 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=12768 command.oublock=32 command.real_time_ms=585 command.spawn_token_fork_ms=1 command.system_time_ms=190 command.user_time_ms=36 component=gitaly.StreamServerInterceptor correlation_id=01KV3AFA2V7XJP22BVM223BSZZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=606 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T15:02:16.985" grpc.time_ms=593.252 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1965vm-test-run-nixbot-gitlab> gitlab # [  198.373136] gitaly[1387]: time="2026-06-14T15:02:17.593Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=621 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV3AFA2V7XJP22BVM223BSZZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-14T15:02:27.267" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T15:02:17.588" grpc.time_ms=5.77 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1966vm-test-run-nixbot-gitlab> gitlab # [  198.450440] gitaly[1387]: time="2026-06-14T15:02:17.701Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFA2V7XJP22BVM223BSZZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-14T15:02:27.025" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T15:02:17.695" grpc.time_ms=5.473 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1967vm-test-run-nixbot-gitlab> gitlab # [  198.463633] gitaly[1387]: time="2026-06-14T15:02:17.725Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV3AFA2V7XJP22BVM223BSZZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-06-14T15:02:27.048" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T15:02:17.719" grpc.time_ms=6.227 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1968vm-test-run-nixbot-gitlab> gitlab # [  198.474636] gitaly[1387]: time="2026-06-14T15:02:17.736Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=658 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV3AFA2V7XJP22BVM223BSZZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-06-14T15:02:47.730" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:17.730" grpc.time_ms=6.083 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1969vm-test-run-nixbot-gitlab> gitlab # [  198.492393] gitaly[1387]: time="2026-06-14T15:02:17.754Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=344 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV3AFA2V7XJP22BVM223BSZZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-06-14T15:02:27.079" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-06-14T15:02:17.749" grpc.time_ms=4.772 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1970vm-test-run-nixbot-gitlab> gitlab # [  198.514671] gitaly[1387]: time="2026-06-14T15:02:17.776Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=4 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV3AFA2V7XJP22BVM223BSZZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-14T15:02:27.099" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T15:02:17.770" grpc.time_ms=6.495 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1971vm-test-run-nixbot-gitlab> gitlab # [  198.523566] gitaly[1387]: time="2026-06-14T15:02:17.785Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KV3AFAVE4TRMDWRD41A0EK84 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=192 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T15:02:17.779" grpc.time_ms=5.468 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc1972vm-test-run-nixbot-gitlab> gitlab # [  198.530325] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:17 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 801973vm-test-run-nixbot-gitlab> gitlab # To http://gitlab/root/test-flake.git1974vm-test-run-nixbot-gitlab> gitlab #    5bc9257..2ee4ce1  master -> master1975vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.34 seconds)1976vm-test-run-nixbot-gitlab> gitlab: must succeed: git -C /tmp/test-flake rev-parse master1977vm-test-run-nixbot-gitlab> gitlab # [  198.620518] gitaly[1387]: time="2026-06-14T15:02:17.880Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFAY24Y036WSP1B4R9P5Z grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T15:02:17.868" pid=1387 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc1978vm-test-run-nixbot-gitlab> gitlab # [  198.621423] gitlab-workhorse[1290]: time="2026-06-14T15:02:17Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671979vm-test-run-nixbot-gitlab> gitlab # [  198.622194] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:17 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 871980vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.02 seconds)1981vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=2ee4ce11810ac90f03a8181f7104629d7f1fa32e'1982vm-test-run-nixbot-gitlab> gitlab # [  198.623263] gitaly[1387]: time="2026-06-14T15:02:17.881Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=5 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFAY24Y036WSP1B4R9P5Z grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T15:02:17.868" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc1983vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=2ee4ce11810ac90f03a8181f7104629d7f1fa32e', in 0.04 seconds)1984vm-test-run-nixbot-gitlab> []1985vm-test-run-nixbot-gitlab> gitlab # [  198.624269] gitaly[1387]: time="2026-06-14T15:02:17.881Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=5 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFAY24Y036WSP1B4R9P5Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T15:02:17.868" grpc.time_ms=13.254 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc1986vm-test-run-nixbot-gitlab> gitlab # [  198.680070] nixbot[3073]: INFO:     ::ffff:127.0.0.1:48648 - "GET /api/repos/gitlab/root/test-flake/builds?commit=2ee4ce11810ac90f03a8181f7104629d7f1fa32e HTTP/1.1" 200 OK1987vm-test-run-nixbot-gitlab> gitlab # [  198.693855] gitaly[1387]: time="2026-06-14T15:02:17.955Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=3 catfile.flush_count=1 catfile.flush_ms=3 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV3AFA2V7XJP22BVM223BSZZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-06-14T15:02:27.000" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:17.951" grpc.time_ms=4.054 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1988vm-test-run-nixbot-gitlab> gitlab # [  198.708054] gitaly[1387]: time="2026-06-14T15:02:17.970Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFB14XRABJDAA6Y9ARMH4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T15:02:17.955" pid=1387 remote_ip="2001:db8:1::1" request_sha=a95112c0eb1098e82891171daa1b6d315505965a response_bytes=383 span.kind=server system=grpc1989vm-test-run-nixbot-gitlab> gitlab # [  198.709440] gitaly[1387]: time="2026-06-14T15:02:17.971Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1210 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFB14XRABJDAA6Y9ARMH4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T15:02:17.955" negotiation.command=fetch negotiation.haves=1 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc1990vm-test-run-nixbot-gitlab> gitlab # [  198.711624] gitaly[1387]: time="2026-06-14T15:02:17.973Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1210 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFB14XRABJDAA6Y9ARMH4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T15:02:17.955" grpc.time_ms=18.167 negotiation.command=fetch negotiation.haves=1 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc1991vm-test-run-nixbot-gitlab> gitlab # [  198.714051] gitlab-workhorse[1290]: time="2026-06-14T15:02:17Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3831992vm-test-run-nixbot-gitlab> gitlab # [  198.714502] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:17 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 383 "" "git/2.54.0" 811993vm-test-run-nixbot-gitlab> gitlab # [  198.783570] gitaly[1387]: time="2026-06-14T15:02:18.045Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFB4KYQPVX34QNX7T1NC4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T15:02:48.044" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:18.044" grpc.time_ms=0.718 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1994vm-test-run-nixbot-gitlab> gitlab # [  198.814547] gitaly[1387]: time="2026-06-14T15:02:18.076Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV3AFB4KYQPVX34QNX7T1NC4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-06-14T15:02:48.069" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T15:02:18.069" grpc.time_ms=6.791 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1995vm-test-run-nixbot-gitlab> gitlab # [  198.841822] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:18.103Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","5bc925784851d57384be462c5a4edebddba61b3c 2ee4ce11810ac90f03a8181f7104629d7f1fa32e refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"8362e7a59589b81eeaa6819b","created_at":"2026-06-14T15:02:17.548Z","trace_propagation_headers":{"sentry-trace":"6f84511efa8f4b92b9b2f3461e9a0170-23fe25ea91474a23","baggage":"sentry-trace_id=6f84511efa8f4b92b9b2f3461e9a0170,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV3AFA2V7XJP22BVM223BSZZ","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-14T15:02:17.550Z","job_size_bytes":185,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-8362e7a59589b81eeaa6819b: done: 0.551116 sec","job_status":"done","queue_duration_s":0.002096,"scheduling_latency_s":0.00217,"gitaly_calls":7,"gitaly_duration_s":0.245863,"redis_calls":112,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.06496,"redis_read_bytes":350,"redis_write_bytes":13069,"redis_cache_calls":9,"redis_cache_duration_s":0.002292,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001189,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":323,"redis_queues_calls":8,"redis_queues_duration_s":0.001193,"redis_queues_read_bytes":8,"redis_queues_write_bytes":6769,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.019712,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1182,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.011657,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.028917,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":29,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":29,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.038,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":29,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.021,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.021,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.038,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.175937,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00074339,"exclusive_lock_hold_duration_s":0.112929716,"duration_s":0.551116,"completed_at":"2026-06-14T15:02:18.103Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.049193,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1996vm-test-run-nixbot-gitlab> gitlab # [  198.852184] gitaly[1387]: time="2026-06-14T15:02:18.112Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV3AFB4KYQPVX34QNX7T1NC4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-14T15:02:28.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T15:02:18.105" grpc.time_ms=6.159 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1997vm-test-run-nixbot-gitlab> gitlab # [  198.947309] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:18.209Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"7fd4a2813794abb8bf15ab51","created_at":"2026-06-14T15:02:17.703Z","trace_propagation_headers":{"sentry-trace":"6f84511efa8f4b92b9b2f3461e9a0170-1884db3002d34e51","baggage":"sentry-trace_id=6f84511efa8f4b92b9b2f3461e9a0170,sentry-release=Unknown"},"correlation_id":"01KV3AFA2V7XJP22BVM223BSZZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:17.705Z","job_size_bytes":46,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-7fd4a2813794abb8bf15ab51: start","job_status":"start","queue_duration_s":0.503626,"scheduling_latency_s":0.503669}1998vm-test-run-nixbot-gitlab> gitlab # [  199.000432] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:18.261Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"7fd4a2813794abb8bf15ab51","created_at":"2026-06-14T15:02:17.703Z","trace_propagation_headers":{"sentry-trace":"6f84511efa8f4b92b9b2f3461e9a0170-1884db3002d34e51","baggage":"sentry-trace_id=6f84511efa8f4b92b9b2f3461e9a0170,sentry-release=Unknown"},"correlation_id":"01KV3AFA2V7XJP22BVM223BSZZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:17.705Z","job_size_bytes":46,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-7fd4a2813794abb8bf15ab51: done: 0.052826 sec","job_status":"done","queue_duration_s":0.503626,"scheduling_latency_s":0.503669,"redis_calls":9,"redis_duration_s":0.003187,"redis_read_bytes":23,"redis_write_bytes":782,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001699,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000444,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001044,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":304,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018731,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.052826,"completed_at":"2026-06-14T15:02:18.261Z","load_balancing_strategy":"primary","db_duration_s":0.001975,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1999vm-test-run-nixbot-gitlab> gitlab # [  199.073527] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:18.335Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"980ec46fdd748d8b980f7683","created_at":"2026-06-14T15:02:17.709Z","trace_propagation_headers":{"sentry-trace":"6f84511efa8f4b92b9b2f3461e9a0170-1884db3002d34e51","baggage":"sentry-trace_id=6f84511efa8f4b92b9b2f3461e9a0170,sentry-release=Unknown"},"correlation_id":"01KV3AFA2V7XJP22BVM223BSZZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:17.711Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-980ec46fdd748d8b980f7683: start","job_status":"start","queue_duration_s":0.623794,"scheduling_latency_s":0.623864}2000vm-test-run-nixbot-gitlab> gitlab # [  199.115232] gitaly[1387]: time="2026-06-14T15:02:18.376Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFB4KYQPVX34QNX7T1NC4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-14T15:02:28.275" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T15:02:18.376" grpc.time_ms=0.514 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2001vm-test-run-nixbot-gitlab> gitlab # [  199.147173] gitaly[1387]: time="2026-06-14T15:02:18.409Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFB4KYQPVX34QNX7T1NC4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-14T15:02:28.295" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T15:02:18.396" grpc.time_ms=12.363 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2002vm-test-run-nixbot-gitlab> gitlab # [  199.196162] gitaly[1387]: time="2026-06-14T15:02:18.458Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFA2V7XJP22BVM223BSZZ diskcache=cf0629c7-5efa-4f2b-9757-9842cb49d9c8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-14T21:02:18.395" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:18.396" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2003vm-test-run-nixbot-gitlab> gitlab # [  199.197313] gitaly[1387]: time="2026-06-14T15:02:18.458Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=5 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1644 command.oublock=0 command.real_time_ms=85 command.spawn_token_fork_ms=3 command.system_time_ms=7 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFA2V7XJP22BVM223BSZZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-14T21:02:18.395" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:18.396" grpc.time_ms=61.487 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2004vm-test-run-nixbot-gitlab> gitlab # [  199.209793] gitaly[1387]: time="2026-06-14T15:02:18.471Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV3AFB4KYQPVX34QNX7T1NC4 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4159 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T15:02:18.471" grpc.time_ms=0.329 pid=1387 span.kind=server system=grpc2005vm-test-run-nixbot-gitlab> gitlab # [  199.266643] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:18.528Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"980ec46fdd748d8b980f7683","created_at":"2026-06-14T15:02:17.709Z","trace_propagation_headers":{"sentry-trace":"6f84511efa8f4b92b9b2f3461e9a0170-1884db3002d34e51","baggage":"sentry-trace_id=6f84511efa8f4b92b9b2f3461e9a0170,sentry-release=Unknown"},"correlation_id":"01KV3AFA2V7XJP22BVM223BSZZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:17.711Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-980ec46fdd748d8b980f7683: done: 0.193666 sec","job_status":"done","queue_duration_s":0.623794,"scheduling_latency_s":0.623864,"gitaly_calls":1,"gitaly_duration_s":0.13853,"redis_calls":12,"redis_duration_s":0.006261,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002447,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001823,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001991,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037658,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.156264274,"duration_s":0.193666,"completed_at":"2026-06-14T15:02:18.528Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007825,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2006vm-test-run-nixbot-gitlab> gitlab # [  199.268905] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:18.531Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"1f3ec1c9eac021990cb804b1","created_at":"2026-06-14T15:02:17.822Z","trace_propagation_headers":{"sentry-trace":"6f84511efa8f4b92b9b2f3461e9a0170-1884db3002d34e51","baggage":"sentry-trace_id=6f84511efa8f4b92b9b2f3461e9a0170,sentry-release=Unknown"},"correlation_id":"01KV3AFA2V7XJP22BVM223BSZZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:abbfaeef9e32e8a43565e0e919df6868c60f4cb227db0bfd87df9f4f2618f559","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:17.824Z","job_size_bytes":467,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-1f3ec1c9eac021990cb804b1: start","job_status":"start","queue_duration_s":0.706483,"scheduling_latency_s":0.706505}2007vm-test-run-nixbot-gitlab> gitlab # [  199.348969] gitaly[1387]: time="2026-06-14T15:02:18.551Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFA2V7XJP22BVM223BSZZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T15:02:48.550" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:18.551" grpc.time_ms=0.605 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2008vm-test-run-nixbot-gitlab> gitlab # [  199.350367] gitaly[1387]: time="2026-06-14T15:02:18.559Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFA2V7XJP22BVM223BSZZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-14T15:02:28.003" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T15:02:18.553" grpc.time_ms=5.34 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2009vm-test-run-nixbot-gitlab> gitlab # [  199.351264] gitaly[1387]: time="2026-06-14T15:02:18.581Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV3AFA2V7XJP22BVM223BSZZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-14T15:02:48.580" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:18.580" grpc.time_ms=0.899 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2010vm-test-run-nixbot-gitlab> gitlab # [  199.421424] gitaly[1387]: time="2026-06-14T15:02:18.586Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=64 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV3AFA2V7XJP22BVM223BSZZ error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-06-14T15:02:48.584" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:18.584" grpc.time_ms=2.057 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2011vm-test-run-nixbot-gitlab> gitlab # [  199.422759] gitaly[1387]: time="2026-06-14T15:02:18.622Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KV3AFA2V7XJP22BVM223BSZZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-14T15:02:28.066" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T15:02:18.616" grpc.time_ms=5.847 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2012vm-test-run-nixbot-gitlab> gitlab # [  199.423896] gitaly[1387]: time="2026-06-14T15:02:18.681Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV3AFB4KYQPVX34QNX7T1NC4 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4161 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T15:02:18.680" grpc.time_ms=0.499 pid=1387 span.kind=server system=grpc2013vm-test-run-nixbot-gitlab> gitlab # [  199.500810] gitaly[1387]: time="2026-06-14T15:02:18.762Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFB4KYQPVX34QNX7T1NC4 diskcache=299b8fcd-140d-44cd-8d88-e552bec67d1e grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-14T15:02:28.004" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T15:02:18.414" pid=1387 ref=refs/keep-around/5bc925784851d57384be462c5a4edebddba61b3c remote_ip="2001:db8:1::1" revision=5bc925784851d57384be462c5a4edebddba61b3c span.kind=server system=grpc username=root2014vm-test-run-nixbot-gitlab> gitlab # [  199.519671] gitaly[1387]: time="2026-06-14T15:02:18.781Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=194 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=856064 command.minflt=5359 command.oublock=8 command.real_time_ms=336 command.spawn_token_fork_ms=5 command.system_time_ms=161 command.user_time_ms=33 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFB4KYQPVX34QNX7T1NC4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-14T15:02:28.004" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T15:02:18.414" grpc.time_ms=348.225 pid=1387 ref=refs/keep-around/5bc925784851d57384be462c5a4edebddba61b3c remote_ip="2001:db8:1::1" revision=5bc925784851d57384be462c5a4edebddba61b3c span.kind=server system=grpc username=root2015vm-test-run-nixbot-gitlab> gitlab # [  199.546679] gitaly[1387]: time="2026-06-14T15:02:18.808Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFB4KYQPVX34QNX7T1NC4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T15:02:48.007" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:18.808" grpc.time_ms=0.611 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2016vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=2ee4ce11810ac90f03a8181f7104629d7f1fa32e'2017vm-test-run-nixbot-gitlab> gitlab # [  199.680364] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:18 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/5bc925784851d57384be462c5a4edebddba61b3c HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 9342018vm-test-run-nixbot-gitlab> gitlab # [  199.682329] nixbot[3073]: {"ts": "2026-06-14T15:02:18.940Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/5bc925784851d57384be462c5a4edebddba61b3c \"HTTP/1.1 201 Created\""}2019vm-test-run-nixbot-gitlab> gitlab # [  199.716426] nixbot[3073]: {"ts": "2026-06-14T15:02:18.978Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-5bc925784851-5b0f65d7", "argv0": "bwrap"}2020vm-test-run-nixbot-gitlab> gitlab # [  199.754759] nixbot[3073]: INFO:     ::ffff:127.0.0.1:48662 - "GET /api/repos/gitlab/root/test-flake/builds?commit=2ee4ce11810ac90f03a8181f7104629d7f1fa32e HTTP/1.1" 200 OK2021vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=2ee4ce11810ac90f03a8181f7104629d7f1fa32e', in 0.08 seconds)2022vm-test-run-nixbot-gitlab> []2023vm-test-run-nixbot-gitlab> gitlab # [  199.771175] gitaly[1387]: time="2026-06-14T15:02:19.032Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=2 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=964 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV3AFA2V7XJP22BVM223BSZZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-14T15:02:49.019" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:19.019" grpc.time_ms=12.805 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2024vm-test-run-nixbot-gitlab> gitlab # [  199.785711] gitaly[1387]: time="2026-06-14T15:02:19.047Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV3AFA2V7XJP22BVM223BSZZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-06-14T15:02:49.037" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:19.037" grpc.time_ms=9.93 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2025vm-test-run-nixbot-gitlab> gitlab # [  199.801198] sidekiq-cluster[2097]: {"severity":"WARN","time":"2026-06-14T15:02:19.063Z","class":"CreatePipelineWorker","correlation_id":"01KV3AFA2V7XJP22BVM223BSZZ","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"5bc925784851d57384be462c5a4edebddba61b3c","after":"2ee4ce11810ac90f03a8181f7104629d7f1fa32e","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"2ee4ce11810ac90f03a8181f7104629d7f1fa32e","ref":"refs/heads/master"},"retry":0}2026vm-test-run-nixbot-gitlab> gitlab # [  199.805731] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:19.067Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"1f3ec1c9eac021990cb804b1","created_at":"2026-06-14T15:02:17.822Z","trace_propagation_headers":{"sentry-trace":"6f84511efa8f4b92b9b2f3461e9a0170-1884db3002d34e51","baggage":"sentry-trace_id=6f84511efa8f4b92b9b2f3461e9a0170,sentry-release=Unknown"},"correlation_id":"01KV3AFA2V7XJP22BVM223BSZZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:abbfaeef9e32e8a43565e0e919df6868c60f4cb227db0bfd87df9f4f2618f559","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:17.824Z","job_size_bytes":467,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-1f3ec1c9eac021990cb804b1: done: 0.536714 sec","job_status":"done","queue_duration_s":0.706483,"scheduling_latency_s":0.706505,"gitaly_calls":7,"gitaly_duration_s":0.053422,"redis_calls":33,"redis_duration_s":0.005213,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000734,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001188,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002324,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000967,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":30,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.065,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":26,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.062,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.40495,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.536714,"completed_at":"2026-06-14T15:02:19.067Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.06402,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2027vm-test-run-nixbot-gitlab> gitlab # [  199.882208] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:19.069Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"6259ec8e-f07d-4223-9a48-26022ab4d6ab\"}"],"class":"WebHookWorker","jid":"4f1864239e7cc184c34e3d92","created_at":"2026-06-14T15:02:17.927Z","trace_propagation_headers":{"sentry-trace":"6f84511efa8f4b92b9b2f3461e9a0170-1884db3002d34e51","baggage":"sentry-trace_id=6f84511efa8f4b92b9b2f3461e9a0170,sentry-release=Unknown"},"correlation_id":"01KV3AFA2V7XJP22BVM223BSZZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f146143138debf3a4511284deb3bd36b1cf17a31b6f4b7659db6f2a8add82159","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:17.934Z","job_size_bytes":1749,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-4f1864239e7cc184c34e3d92: start","job_status":"start","queue_duration_s":1.134601,"scheduling_latency_s":1.134628}2028vm-test-run-nixbot-gitlab> gitlab # [  199.935123] nixbot[3073]: INFO:     ::ffff:127.0.0.1:48664 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted2029vm-test-run-nixbot-gitlab> gitlab # [  199.958175] systemd[1]: Started Nix Daemon.2030vm-test-run-nixbot-gitlab> gitlab # [  199.966795] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:19.228Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"6259ec8e-f07d-4223-9a48-26022ab4d6ab\"}"],"class":"WebHookWorker","jid":"4f1864239e7cc184c34e3d92","created_at":"2026-06-14T15:02:17.927Z","trace_propagation_headers":{"sentry-trace":"6f84511efa8f4b92b9b2f3461e9a0170-1884db3002d34e51","baggage":"sentry-trace_id=6f84511efa8f4b92b9b2f3461e9a0170,sentry-release=Unknown"},"correlation_id":"01KV3AFA2V7XJP22BVM223BSZZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f146143138debf3a4511284deb3bd36b1cf17a31b6f4b7659db6f2a8add82159","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:17.934Z","job_size_bytes":1749,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-4f1864239e7cc184c34e3d92: done: 0.159328 sec","job_status":"done","queue_duration_s":1.134601,"scheduling_latency_s":1.134628,"redis_calls":19,"redis_duration_s":0.016855000000000002,"redis_read_bytes":31,"redis_write_bytes":5281,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000875,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.001837,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3643,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.008272,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.005871,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.07329060000000709,"cpu_s":0.058279,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000520457,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","duration_s":0.159328,"completed_at":"2026-06-14T15:02:19.228Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003509,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2031vm-test-run-nixbot-gitlab> gitlab # [  200.002809] nix-daemon[3369]: accepted connection from pid 3363, user nixbot2032vm-test-run-nixbot-gitlab> gitlab # [  200.007645] nix-daemon[3369]: accepted connection from pid 3361, user nixbot2033vm-test-run-nixbot-gitlab> gitlab # [  200.052453] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:19.314Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"e86683c4c2e159830bba2b3a","created_at":"2026-06-14T15:02:18.814Z","trace_propagation_headers":{"sentry-trace":"ce4cab28ec704bc6bdf3b972d8e39104-d8607a23537f4669","baggage":"sentry-trace_id=ce4cab28ec704bc6bdf3b972d8e39104,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV3AFB4KYQPVX34QNX7T1NC4","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:18.820Z","job_size_bytes":65,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-e86683c4c2e159830bba2b3a: start","job_status":"start","queue_duration_s":0.493117,"scheduling_latency_s":0.493157}2034vm-test-run-nixbot-gitlab> gitlab # [  200.065676] gitaly[1387]: time="2026-06-14T15:02:19.327Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV3AFCAR2FNY9HH9ZJTBZE6P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T15:02:19.320" grpc.time_ms=7.176 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc2035vm-test-run-nixbot-gitlab> gitlab # [  200.067074] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:19 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 1032036vm-test-run-nixbot-gitlab> gitlab # [  200.126783] gitaly[1387]: time="2026-06-14T15:02:19.389Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFCE1J0753GK3ZVTSG3PS grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T15:02:19.380" pid=1387 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc2037vm-test-run-nixbot-gitlab> gitlab # [  200.127417] gitaly[1387]: time="2026-06-14T15:02:19.389Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFCE1J0753GK3ZVTSG3PS grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T15:02:19.380" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc2038vm-test-run-nixbot-gitlab> gitlab # [  200.132224] gitaly[1387]: time="2026-06-14T15:02:19.394Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFCE1J0753GK3ZVTSG3PS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-14T15:02:19.380" grpc.time_ms=13.175 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc2039vm-test-run-nixbot-gitlab> gitlab # [  200.134371] gitlab-workhorse[1290]: time="2026-06-14T15:02:19Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672040vm-test-run-nixbot-gitlab> gitlab # [  200.135962] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:19 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 662041vm-test-run-nixbot-gitlab> gitlab # [  200.200318] nixbot[3073]: {"ts": "2026-06-14T15:02:19.462Z", "level": "info", "logger": "nixbot.canceller", "msg": "cancelling build", "build_id": 1, "reason": "superseded"}2042vm-test-run-nixbot-gitlab> gitlab # [  200.205670] nix-daemon[3374]: unexpected Nix daemon error: error: interrupted by the user2043vm-test-run-nixbot-gitlab> gitlab # [  200.242516] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:19.504Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"e86683c4c2e159830bba2b3a","created_at":"2026-06-14T15:02:18.814Z","trace_propagation_headers":{"sentry-trace":"ce4cab28ec704bc6bdf3b972d8e39104-d8607a23537f4669","baggage":"sentry-trace_id=ce4cab28ec704bc6bdf3b972d8e39104,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV3AFB4KYQPVX34QNX7T1NC4","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:18.820Z","job_size_bytes":65,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-e86683c4c2e159830bba2b3a: done: 0.190438 sec","job_status":"done","queue_duration_s":0.493117,"scheduling_latency_s":0.493157,"redis_calls":18,"redis_duration_s":0.076056,"redis_read_bytes":271,"redis_write_bytes":2103,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.073692,"redis_feature_flag_read_bytes":264,"redis_feature_flag_write_bytes":1285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001745,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000619,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029859,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000516266,"duration_s":0.190438,"completed_at":"2026-06-14T15:02:19.504Z","load_balancing_strategy":"primary","db_duration_s":0.01541,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2044vm-test-run-nixbot-gitlab> gitlab # [  200.255226] gitaly[1387]: time="2026-06-14T15:02:19.517Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFCJ9XJJ8HRSH21SEYHPD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T15:02:49.416" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:19.516" grpc.time_ms=0.553 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2045vm-test-run-nixbot-gitlab> gitlab # [  200.278976] gitaly[1387]: time="2026-06-14T15:02:19.541Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=647 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV3AFCJ9XJJ8HRSH21SEYHPD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-06-14T15:02:49.433" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T15:02:19.534" grpc.time_ms=6.372 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2046vm-test-run-nixbot-gitlab> gitlab # [  200.351057] gitaly[1387]: time="2026-06-14T15:02:19.554Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV3AFCJ9XJJ8HRSH21SEYHPD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-14T15:02:29.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T15:02:19.549" grpc.time_ms=4.945 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2047vm-test-run-nixbot-gitlab> gitlab # [  200.352388] gitaly[1387]: time="2026-06-14T15:02:19.589Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFCJ9XJJ8HRSH21SEYHPD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-14T15:02:29.048" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T15:02:19.589" grpc.time_ms=0.2 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2048vm-test-run-nixbot-gitlab> gitlab # [  200.353282] gitaly[1387]: time="2026-06-14T15:02:19.599Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFCJ9XJJ8HRSH21SEYHPD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-14T15:02:29.054" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T15:02:19.594" grpc.time_ms=5.243 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2049vm-test-run-nixbot-gitlab> gitlab # [  200.393942] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:19.655Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"1fd2576373841cbd955dff7e","created_at":"2026-06-14T15:02:18.822Z","trace_propagation_headers":{"sentry-trace":"ce4cab28ec704bc6bdf3b972d8e39104-d8607a23537f4669","baggage":"sentry-trace_id=ce4cab28ec704bc6bdf3b972d8e39104,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV3AFB4KYQPVX34QNX7T1NC4","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:18.825Z","job_size_bytes":65,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-1fd2576373841cbd955dff7e: start","job_status":"start","queue_duration_s":0.830059,"scheduling_latency_s":0.830097}2050vm-test-run-nixbot-gitlab> gitlab # [  200.402084] gitaly[1387]: time="2026-06-14T15:02:19.663Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV3AFCJ9XJJ8HRSH21SEYHPD grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4159 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T15:02:19.662" grpc.time_ms=0.35 pid=1387 span.kind=server system=grpc2051vm-test-run-nixbot-gitlab> gitlab # [  200.428088] gitaly[1387]: time="2026-06-14T15:02:19.689Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV3AFCJ9XJJ8HRSH21SEYHPD grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4161 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T15:02:19.688" grpc.time_ms=0.27 pid=1387 span.kind=server system=grpc2052vm-test-run-nixbot-gitlab> gitlab # [  200.434527] gitaly[1387]: time="2026-06-14T15:02:19.696Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFCJYRNKS6M1EABQ9HWJH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T15:02:49.695" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:19.696" grpc.time_ms=0.441 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2053vm-test-run-nixbot-gitlab> gitlab # [  200.438665] gitaly[1387]: time="2026-06-14T15:02:19.700Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFCJ9XJJ8HRSH21SEYHPD diskcache=baceda53-317c-454d-abb6-1719c9a0881c grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-14T15:02:29.061" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T15:02:19.601" pid=1387 ref=refs/keep-around/2ee4ce11810ac90f03a8181f7104629d7f1fa32e remote_ip="2001:db8:1::1" revision=2ee4ce11810ac90f03a8181f7104629d7f1fa32e span.kind=server system=grpc username=root2054vm-test-run-nixbot-gitlab> gitlab # [  200.515544] gitaly[1387]: time="2026-06-14T15:02:19.701Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=53 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4983 command.oublock=8 command.real_time_ms=90 command.spawn_token_fork_ms=0 command.system_time_ms=37 command.user_time_ms=16 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFCJ9XJJ8HRSH21SEYHPD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-14T15:02:29.061" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-14T15:02:19.601" grpc.time_ms=99.367 pid=1387 ref=refs/keep-around/2ee4ce11810ac90f03a8181f7104629d7f1fa32e remote_ip="2001:db8:1::1" revision=2ee4ce11810ac90f03a8181f7104629d7f1fa32e span.kind=server system=grpc username=root2055vm-test-run-nixbot-gitlab> gitlab # [  200.516835] nixbot[3073]: {"ts": "2026-06-14T15:02:19.771Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/5bc925784851d57384be462c5a4edebddba61b3c \"HTTP/1.1 201 Created\""}2056vm-test-run-nixbot-gitlab> gitlab # [  200.518830] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:19.723Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"1fd2576373841cbd955dff7e","created_at":"2026-06-14T15:02:18.822Z","trace_propagation_headers":{"sentry-trace":"ce4cab28ec704bc6bdf3b972d8e39104-d8607a23537f4669","baggage":"sentry-trace_id=ce4cab28ec704bc6bdf3b972d8e39104,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV3AFB4KYQPVX34QNX7T1NC4","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:18.825Z","job_size_bytes":65,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-1fd2576373841cbd955dff7e: done: 0.067247 sec","job_status":"done","queue_duration_s":0.830059,"scheduling_latency_s":0.830097,"redis_calls":15,"redis_duration_s":0.006137,"redis_read_bytes":142,"redis_write_bytes":1431,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000849,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":431,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002902,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002386,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035835,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000490565,"duration_s":0.067247,"completed_at":"2026-06-14T15:02:19.723Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012113,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2057vm-test-run-nixbot-gitlab> gitlab # [  200.603076] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:19.724Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"376708f5331255fc847b02a3","created_at":"2026-06-14T15:02:18.917Z","trace_propagation_headers":{"sentry-trace":"ce4cab28ec704bc6bdf3b972d8e39104-d8607a23537f4669","baggage":"sentry-trace_id=ce4cab28ec704bc6bdf3b972d8e39104,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV3AFB4KYQPVX34QNX7T1NC4","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:18.923Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-376708f5331255fc847b02a3: start","job_status":"start","queue_duration_s":0.801278,"scheduling_latency_s":0.801301}2058vm-test-run-nixbot-gitlab> gitlab # [  200.627375] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:19 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/5bc925784851d57384be462c5a4edebddba61b3c HTTP/1.1" 201 651 "" "python-httpx/0.28.1" 2842059vm-test-run-nixbot-gitlab> gitlab # [  200.628749] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:19 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/2ee4ce11810ac90f03a8181f7104629d7f1fa32e HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 3922060vm-test-run-nixbot-gitlab> gitlab # [  200.629861] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:19 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/5bc925784851d57384be462c5a4edebddba61b3c HTTP/1.1" 201 665 "" "python-httpx/0.28.1" 1112061vm-test-run-nixbot-gitlab> gitlab # [  200.631130] gitaly[1387]: time="2026-06-14T15:02:19.709Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFCJ9XJJ8HRSH21SEYHPD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T15:02:49.607" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:19.708" grpc.time_ms=1.449 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2062vm-test-run-nixbot-gitlab> gitlab # [  200.631987] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:19.850Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"85fe7d19e856493a471c2a5e","created_at":"2026-06-14T15:02:19.848Z","trace_propagation_headers":{"sentry-trace":"ce4cab28ec704bc6bdf3b972d8e39104-a9048d97b48d4638","baggage":"sentry-trace_id=ce4cab28ec704bc6bdf3b972d8e39104,sentry-release=Unknown"},"correlation_id":"01KV3AFB4KYQPVX34QNX7T1NC4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","duplicate-of":"376708f5331255fc847b02a3","job_size_bytes":3,"pid":2097,"job_status":"deduplicated","message":"PipelineProcessWorker JID-85fe7d19e856493a471c2a5e: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}2063vm-test-run-nixbot-gitlab> gitlab # [  200.632519] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:19.852Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"376708f5331255fc847b02a3","created_at":"2026-06-14T15:02:18.917Z","trace_propagation_headers":{"sentry-trace":"ce4cab28ec704bc6bdf3b972d8e39104-d8607a23537f4669","baggage":"sentry-trace_id=ce4cab28ec704bc6bdf3b972d8e39104,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV3AFB4KYQPVX34QNX7T1NC4","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:18.923Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2097,"job_status":"rescheduled","message":"PipelineProcessWorker JID-376708f5331255fc847b02a3: rescheduled"}2064vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=2ee4ce11810ac90f03a8181f7104629d7f1fa32e'2065vm-test-run-nixbot-gitlab> gitlab # [  200.720284] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:19.859Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"376708f5331255fc847b02a3","created_at":"2026-06-14T15:02:18.917Z","trace_propagation_headers":{"sentry-trace":"ce4cab28ec704bc6bdf3b972d8e39104-d8607a23537f4669","baggage":"sentry-trace_id=ce4cab28ec704bc6bdf3b972d8e39104,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV3AFB4KYQPVX34QNX7T1NC4","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:18.923Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-376708f5331255fc847b02a3: done: 0.134303 sec","job_status":"done","queue_duration_s":0.801278,"scheduling_latency_s":0.801301,"gitaly_calls":3,"gitaly_duration_s":0.007797,"redis_calls":54,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.014950999999999999,"redis_read_bytes":292,"redis_write_bytes":8550,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001055,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":16,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.00165,"redis_cache_read_bytes":32,"redis_cache_write_bytes":1470,"redis_feature_flag_calls":7,"redis_feature_flag_duration_s":0.002681,"redis_feature_flag_read_bytes":134,"redis_feature_flag_write_bytes":992,"redis_queues_calls":4,"redis_queues_duration_s":0.000558,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2220,"redis_queues_metadata_calls":12,"redis_queues_metadata_duration_s":0.004351,"redis_queues_metadata_read_bytes":89,"redis_queues_metadata_write_bytes":1500,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.00053,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000528,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002543,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":34,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":34,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":22,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.089765,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.100056013,"duration_s":0.134303,"completed_at":"2026-06-14T15:02:19.859Z","load_balancing_strategy":"primary","db_duration_s":0.024173,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2066vm-test-run-nixbot-gitlab> gitlab # [  200.742838] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:19.860Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","4f1864239e7cc184c34e3d92"],"class":"WebHooks::LogExecutionWorker","jid":"cea6437d25a9075f19bded45","created_at":"2026-06-14T15:02:19.201Z","trace_propagation_headers":{"sentry-trace":"6f84511efa8f4b92b9b2f3461e9a0170-22a3a23a34fe4eba","baggage":"sentry-trace_id=6f84511efa8f4b92b9b2f3461e9a0170,sentry-release=Unknown"},"correlation_id":"01KV3AFA2V7XJP22BVM223BSZZ","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c15b7750f90788ef96df4441d64474b3db5798b6a0e80d572cf490a3d85e7a2c","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:19.209Z","job_size_bytes":2506,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-cea6437d25a9075f19bded45: start","job_status":"start","queue_duration_s":0.65179,"scheduling_latency_s":0.651811}2067vm-test-run-nixbot-gitlab> gitlab # [  200.813571] nixbot[3073]: {"ts": "2026-06-14T15:02:19.859Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/2ee4ce11810ac90f03a8181f7104629d7f1fa32e \"HTTP/1.1 201 Created\""}2068vm-test-run-nixbot-gitlab> gitlab # [  200.814375] nixbot[3073]: {"ts": "2026-06-14T15:02:19.869Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-2ee4ce11810a-0b9310b1", "argv0": "bwrap"}2069vm-test-run-nixbot-gitlab> gitlab # [  200.815577] gitaly[1387]: time="2026-06-14T15:02:19.731Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KV3AFCJYRNKS6M1EABQ9HWJH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-14T15:02:29.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T15:02:19.725" grpc.time_ms=5.807 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2070vm-test-run-nixbot-gitlab> gitlab # [  200.817123] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:19.955Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","4f1864239e7cc184c34e3d92"],"class":"WebHooks::LogExecutionWorker","jid":"cea6437d25a9075f19bded45","created_at":"2026-06-14T15:02:19.201Z","trace_propagation_headers":{"sentry-trace":"6f84511efa8f4b92b9b2f3461e9a0170-22a3a23a34fe4eba","baggage":"sentry-trace_id=6f84511efa8f4b92b9b2f3461e9a0170,sentry-release=Unknown"},"correlation_id":"01KV3AFA2V7XJP22BVM223BSZZ","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c15b7750f90788ef96df4441d64474b3db5798b6a0e80d572cf490a3d85e7a2c","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:19.209Z","job_size_bytes":2506,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-cea6437d25a9075f19bded45: done: 0.094508 sec","job_status":"done","queue_duration_s":0.65179,"scheduling_latency_s":0.651811,"redis_calls":10,"redis_duration_s":0.00252,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00047,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001504,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000546,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.022,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.022,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.080112,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000358146,"duration_s":0.094508,"completed_at":"2026-06-14T15:02:19.955Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006321,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2071vm-test-run-nixbot-gitlab> gitlab # [  200.888376] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:20.017Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"38f4b5159009069ed05a1bc9","created_at":"2026-06-14T15:02:19.715Z","trace_propagation_headers":{"sentry-trace":"8c1f8065846446abbf793fe6569066d8-213246d80b05483c","baggage":"sentry-trace_id=8c1f8065846446abbf793fe6569066d8,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV3AFCJ9XJJ8HRSH21SEYHPD","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:19.721Z","job_size_bytes":65,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-38f4b5159009069ed05a1bc9: start","job_status":"start","queue_duration_s":0.295931,"scheduling_latency_s":0.295968}2072vm-test-run-nixbot-gitlab> gitlab # [  200.897897] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:20.039Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"38f4b5159009069ed05a1bc9","created_at":"2026-06-14T15:02:19.715Z","trace_propagation_headers":{"sentry-trace":"8c1f8065846446abbf793fe6569066d8-213246d80b05483c","baggage":"sentry-trace_id=8c1f8065846446abbf793fe6569066d8,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV3AFCJ9XJJ8HRSH21SEYHPD","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:19.721Z","job_size_bytes":65,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-38f4b5159009069ed05a1bc9: done: 0.022541 sec","job_status":"done","queue_duration_s":0.295931,"scheduling_latency_s":0.295968,"redis_calls":7,"redis_duration_s":0.00161,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00118,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00043,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014359,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022541,"completed_at":"2026-06-14T15:02:20.039Z","load_balancing_strategy":"primary","db_duration_s":0.002604,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2073vm-test-run-nixbot-gitlab> gitlab # [  200.981142] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:20.042Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"6548c32e34646f01b2377529","created_at":"2026-06-14T15:02:19.722Z","trace_propagation_headers":{"sentry-trace":"8c1f8065846446abbf793fe6569066d8-213246d80b05483c","baggage":"sentry-trace_id=8c1f8065846446abbf793fe6569066d8,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV3AFCJ9XJJ8HRSH21SEYHPD","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:19.725Z","job_size_bytes":65,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-6548c32e34646f01b2377529: start","job_status":"start","queue_duration_s":0.316881,"scheduling_latency_s":0.316918}2074vm-test-run-nixbot-gitlab> gitlab # [  201.057320] gitaly[1387]: time="2026-06-14T15:02:19.800Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV3AFB4KYQPVX34QNX7T1NC4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-14T15:02:49.799" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:19.799" grpc.time_ms=0.993 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2075vm-test-run-nixbot-gitlab> gitlab # [  201.058912] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:20.083Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"6548c32e34646f01b2377529","created_at":"2026-06-14T15:02:19.722Z","trace_propagation_headers":{"sentry-trace":"8c1f8065846446abbf793fe6569066d8-213246d80b05483c","baggage":"sentry-trace_id=8c1f8065846446abbf793fe6569066d8,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV3AFCJ9XJJ8HRSH21SEYHPD","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:19.725Z","job_size_bytes":65,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-6548c32e34646f01b2377529: done: 0.040993 sec","job_status":"done","queue_duration_s":0.316881,"scheduling_latency_s":0.316918,"redis_calls":10,"redis_duration_s":0.002404,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001231,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.001173,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026468,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.040993,"completed_at":"2026-06-14T15:02:20.083Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008252,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2076vm-test-run-nixbot-gitlab> gitlab # [  201.150169] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:20.085Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"092a008a3f7886da67ed6739","created_at":"2026-06-14T15:02:19.785Z","trace_propagation_headers":{"sentry-trace":"ce4cab28ec704bc6bdf3b972d8e39104-a9048d97b48d4638","baggage":"sentry-trace_id=ce4cab28ec704bc6bdf3b972d8e39104,sentry-release=Unknown"},"correlation_id":"01KV3AFB4KYQPVX34QNX7T1NC4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:19.789Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-092a008a3f7886da67ed6739: start","job_status":"start","queue_duration_s":0.295534,"scheduling_latency_s":0.295559}2077vm-test-run-nixbot-gitlab> gitlab # [  201.151280] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:20.111Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"092a008a3f7886da67ed6739","created_at":"2026-06-14T15:02:19.785Z","trace_propagation_headers":{"sentry-trace":"ce4cab28ec704bc6bdf3b972d8e39104-a9048d97b48d4638","baggage":"sentry-trace_id=ce4cab28ec704bc6bdf3b972d8e39104,sentry-release=Unknown"},"correlation_id":"01KV3AFB4KYQPVX34QNX7T1NC4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:19.789Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-092a008a3f7886da67ed6739: done: 0.026712 sec","job_status":"done","queue_duration_s":0.295534,"scheduling_latency_s":0.295559,"redis_calls":10,"redis_duration_s":0.002182,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000415,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00131,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000457,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018337,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000326857,"duration_s":0.026712,"completed_at":"2026-06-14T15:02:20.111Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003044,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2078vm-test-run-nixbot-gitlab> gitlab # [  201.231167] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:20.113Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"c9f77915802decec7e2dff42","created_at":"2026-06-14T15:02:19.836Z","trace_propagation_headers":{"sentry-trace":"8c1f8065846446abbf793fe6569066d8-213246d80b05483c","baggage":"sentry-trace_id=8c1f8065846446abbf793fe6569066d8,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV3AFCJ9XJJ8HRSH21SEYHPD","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:19.840Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-c9f77915802decec7e2dff42: start","job_status":"start","queue_duration_s":0.272622,"scheduling_latency_s":0.272642}2079vm-test-run-nixbot-gitlab> gitlab # [  201.232773] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:20.223Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"c9f77915802decec7e2dff42","created_at":"2026-06-14T15:02:19.836Z","trace_propagation_headers":{"sentry-trace":"8c1f8065846446abbf793fe6569066d8-213246d80b05483c","baggage":"sentry-trace_id=8c1f8065846446abbf793fe6569066d8,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV3AFCJ9XJJ8HRSH21SEYHPD","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:19.840Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-c9f77915802decec7e2dff42: done: 0.110591 sec","job_status":"done","queue_duration_s":0.272622,"scheduling_latency_s":0.272642,"gitaly_calls":3,"gitaly_duration_s":0.008626,"redis_calls":44,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.0077,"redis_read_bytes":98,"redis_write_bytes":6013,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001162,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.001774,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":2,"redis_queues_duration_s":0.000287,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1132,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001668,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000244,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000446,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000957,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":31,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":31,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":19,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.077914,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.092806488,"duration_s":0.110591,"completed_at":"2026-06-14T15:02:20.223Z","load_balancing_strategy":"primary","db_duration_s":0.022221,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2080vm-test-run-nixbot-gitlab> gitlab # [  201.311139] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:20.225Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"213ba772dded380717596eac","created_at":"2026-06-14T15:02:19.852Z","trace_propagation_headers":{"sentry-trace":"ce4cab28ec704bc6bdf3b972d8e39104-a9048d97b48d4638","baggage":"sentry-trace_id=ce4cab28ec704bc6bdf3b972d8e39104,sentry-release=Unknown"},"correlation_id":"01KV3AFB4KYQPVX34QNX7T1NC4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:19.853Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-213ba772dded380717596eac: start","job_status":"start","queue_duration_s":0.371899,"scheduling_latency_s":0.371923}2081vm-test-run-nixbot-gitlab> gitlab # [  201.318345] gitaly[1387]: time="2026-06-14T15:02:19.823Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFCW0TMEDD58CW2Y1WZZZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T15:02:49.822" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:19.823" grpc.time_ms=0.571 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2082vm-test-run-nixbot-gitlab> gitlab # [  201.388107] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:20.483Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"213ba772dded380717596eac","created_at":"2026-06-14T15:02:19.852Z","trace_propagation_headers":{"sentry-trace":"ce4cab28ec704bc6bdf3b972d8e39104-a9048d97b48d4638","baggage":"sentry-trace_id=ce4cab28ec704bc6bdf3b972d8e39104,sentry-release=Unknown"},"correlation_id":"01KV3AFB4KYQPVX34QNX7T1NC4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:19.853Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2097,"job_status":"rescheduled","message":"PipelineProcessWorker JID-213ba772dded380717596eac: rescheduled"}2083vm-test-run-nixbot-gitlab> gitlab # [  201.389356] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:20.489Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"213ba772dded380717596eac","created_at":"2026-06-14T15:02:19.852Z","trace_propagation_headers":{"sentry-trace":"ce4cab28ec704bc6bdf3b972d8e39104-a9048d97b48d4638","baggage":"sentry-trace_id=ce4cab28ec704bc6bdf3b972d8e39104,sentry-release=Unknown"},"correlation_id":"01KV3AFB4KYQPVX34QNX7T1NC4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:19.853Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-213ba772dded380717596eac: done: 0.264132 sec","job_status":"done","queue_duration_s":0.371899,"scheduling_latency_s":0.371923,"gitaly_calls":4,"gitaly_duration_s":0.089706,"redis_calls":101,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.019168,"redis_read_bytes":185,"redis_write_bytes":23554,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000976,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":17,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.001781,"redis_cache_read_bytes":33,"redis_cache_write_bytes":1526,"redis_feature_flag_calls":22,"redis_feature_flag_duration_s":0.005126,"redis_feature_flag_read_bytes":22,"redis_feature_flag_write_bytes":3871,"redis_queues_calls":22,"redis_queues_duration_s":0.002954,"redis_queues_read_bytes":25,"redis_queues_write_bytes":12386,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.004619,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.00023,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00058,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001926,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":850,"db_count":47,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.01,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.013,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.014,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.131608,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.242673022,"duration_s":0.264132,"completed_at":"2026-06-14T15:02:20.489Z","load_balancing_strategy":"primary","db_duration_s":0.03428,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2084vm-test-run-nixbot-gitlab> gitlab # [  201.471294] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:20.490Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"aa34961e11d692d72c0209fb","created_at":"2026-06-14T15:02:20.158Z","trace_propagation_headers":{"sentry-trace":"8c1f8065846446abbf793fe6569066d8-b6ccb72c8b3b45a8","baggage":"sentry-trace_id=8c1f8065846446abbf793fe6569066d8,sentry-release=Unknown"},"correlation_id":"01KV3AFCJ9XJJ8HRSH21SEYHPD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:20.159Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-aa34961e11d692d72c0209fb: start","job_status":"start","queue_duration_s":0.330964,"scheduling_latency_s":0.330987}2085vm-test-run-nixbot-gitlab> gitlab # [  201.473122] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:20.514Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"aa34961e11d692d72c0209fb","created_at":"2026-06-14T15:02:20.158Z","trace_propagation_headers":{"sentry-trace":"8c1f8065846446abbf793fe6569066d8-b6ccb72c8b3b45a8","baggage":"sentry-trace_id=8c1f8065846446abbf793fe6569066d8,sentry-release=Unknown"},"correlation_id":"01KV3AFCJ9XJJ8HRSH21SEYHPD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:20.159Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-aa34961e11d692d72c0209fb: done: 0.023791 sec","job_status":"done","queue_duration_s":0.330964,"scheduling_latency_s":0.330987,"redis_calls":7,"redis_duration_s":0.001671,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001205,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000466,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016362,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023791,"completed_at":"2026-06-14T15:02:20.514Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00297,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2086vm-test-run-nixbot-gitlab> gitlab # [  201.550337] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:20.516Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"7434aed75db4c139c719a884","created_at":"2026-06-14T15:02:20.275Z","trace_propagation_headers":{"sentry-trace":"ce4cab28ec704bc6bdf3b972d8e39104-72552e6a23414af2","baggage":"sentry-trace_id=ce4cab28ec704bc6bdf3b972d8e39104,sentry-release=Unknown"},"correlation_id":"01KV3AFB4KYQPVX34QNX7T1NC4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:20.278Z","job_size_bytes":23,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-7434aed75db4c139c719a884: start","job_status":"start","queue_duration_s":0.237302,"scheduling_latency_s":0.237323}2087vm-test-run-nixbot-gitlab> gitlab # [  201.620375] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:20.598Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"7434aed75db4c139c719a884","created_at":"2026-06-14T15:02:20.275Z","trace_propagation_headers":{"sentry-trace":"ce4cab28ec704bc6bdf3b972d8e39104-72552e6a23414af2","baggage":"sentry-trace_id=ce4cab28ec704bc6bdf3b972d8e39104,sentry-release=Unknown"},"correlation_id":"01KV3AFB4KYQPVX34QNX7T1NC4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:20.278Z","job_size_bytes":23,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-7434aed75db4c139c719a884: done: 0.082862 sec","job_status":"done","queue_duration_s":0.237302,"scheduling_latency_s":0.237323,"redis_calls":15,"redis_duration_s":0.003276,"redis_read_bytes":13,"redis_write_bytes":3011,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000415,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_calls":2,"redis_queues_duration_s":0.000326,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1626,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001558,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":742,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000479,"redis_rate_limiting_write_bytes":148,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000498,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":13,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":13,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.065329,"worker_id":"sidekiq_0","rate_limiting_gates":["permanent_email_failure","temporary_email_failure"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000335238,"duration_s":0.082862,"completed_at":"2026-06-14T15:02:20.598Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011325,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2088vm-test-run-nixbot-gitlab> gitlab # [  201.696110] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:20.600Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"57ec40147dc3dbc90e0fbc90","created_at":"2026-06-14T15:02:20.590Z","trace_propagation_headers":{"sentry-trace":"ce4cab28ec704bc6bdf3b972d8e39104-88126ae51056466a","baggage":"sentry-trace_id=ce4cab28ec704bc6bdf3b972d8e39104,sentry-release=Unknown"},"correlation_id":"01KV3AFB4KYQPVX34QNX7T1NC4","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:9e616bcd7a0f8020fe4a71f144b0c35ac112fbe8a64aab2db00fccf6da5206b9","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:20.592Z","job_size_bytes":464,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-57ec40147dc3dbc90e0fbc90: start","job_status":"start","queue_duration_s":0.008105,"scheduling_latency_s":0.008126}2089vm-test-run-nixbot-gitlab> gitlab # [  201.697521] gitaly[1387]: time="2026-06-14T15:02:19.827Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFB4KYQPVX34QNX7T1NC4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T15:02:49.826" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:19.826" grpc.time_ms=0.724 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2090vm-test-run-nixbot-gitlab> gitlab # [  201.698500] sidekiq-cluster[2097]: {"severity":"WARN","time":"2026-06-14T15:02:20.631Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"57ec40147dc3dbc90e0fbc90","created_at":"2026-06-14T15:02:20.590Z","trace_propagation_headers":{"sentry-trace":"ce4cab28ec704bc6bdf3b972d8e39104-88126ae51056466a","baggage":"sentry-trace_id=ce4cab28ec704bc6bdf3b972d8e39104,sentry-release=Unknown"},"correlation_id":"01KV3AFB4KYQPVX34QNX7T1NC4","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:9e616bcd7a0f8020fe4a71f144b0c35ac112fbe8a64aab2db00fccf6da5206b9","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:20.592Z","job_size_bytes":464,"pid":2097,"sidekiq_tid":"m3wt","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-57ec40147dc3dbc90e0fbc90: fail: 0.030698 sec","job_status":"fail","queue_duration_s":0.008105,"scheduling_latency_s":0.008126,"redis_calls":7,"redis_duration_s":0.001563,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001112,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000451,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019816,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030698,"completed_at":"2026-06-14T15:02:20.630Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/pipelines.rb:33:in `pipeline_mail'","app/mailers/emails/pipelines.rb:10:in `pipeline_failed_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.00461,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2091vm-test-run-nixbot-gitlab> gitlab # [  202.336086] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:20.634Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"65fc821288f4c076693d939f","created_at":"2026-06-14T15:02:20.371Z","trace_propagation_headers":{"sentry-trace":"ce4cab28ec704bc6bdf3b972d8e39104-72552e6a23414af2","baggage":"sentry-trace_id=ce4cab28ec704bc6bdf3b972d8e39104,sentry-release=Unknown"},"correlation_id":"01KV3AFB4KYQPVX34QNX7T1NC4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:20.375Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-65fc821288f4c076693d939f: start","job_status":"start","queue_duration_s":0.258405,"scheduling_latency_s":0.258446}2092vm-test-run-nixbot-gitlab> gitlab # [  202.337323] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:20.699Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"65fc821288f4c076693d939f","created_at":"2026-06-14T15:02:20.371Z","trace_propagation_headers":{"sentry-trace":"ce4cab28ec704bc6bdf3b972d8e39104-72552e6a23414af2","baggage":"sentry-trace_id=ce4cab28ec704bc6bdf3b972d8e39104,sentry-release=Unknown"},"correlation_id":"01KV3AFB4KYQPVX34QNX7T1NC4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:20.375Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-65fc821288f4c076693d939f: done: 0.064974 sec","job_status":"done","queue_duration_s":0.258405,"scheduling_latency_s":0.258446,"redis_calls":10,"redis_duration_s":0.002385,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000501,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001435,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000449,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.041,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.038,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016151,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000391669,"duration_s":0.064974,"completed_at":"2026-06-14T15:02:20.699Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.04097,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2093vm-test-run-nixbot-gitlab> gitlab # [  202.393136] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:20.701Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"8ee9d58407d6c77532b674ca","created_at":"2026-06-14T15:02:20.386Z","trace_propagation_headers":{"sentry-trace":"ce4cab28ec704bc6bdf3b972d8e39104-72552e6a23414af2","baggage":"sentry-trace_id=ce4cab28ec704bc6bdf3b972d8e39104,sentry-release=Unknown"},"correlation_id":"01KV3AFB4KYQPVX34QNX7T1NC4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:20.391Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-8ee9d58407d6c77532b674ca: start","job_status":"start","queue_duration_s":0.309927,"scheduling_latency_s":0.309949}2094vm-test-run-nixbot-gitlab> gitlab # [  202.418846] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:20.730Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"8ee9d58407d6c77532b674ca","created_at":"2026-06-14T15:02:20.386Z","trace_propagation_headers":{"sentry-trace":"ce4cab28ec704bc6bdf3b972d8e39104-72552e6a23414af2","baggage":"sentry-trace_id=ce4cab28ec704bc6bdf3b972d8e39104,sentry-release=Unknown"},"correlation_id":"01KV3AFB4KYQPVX34QNX7T1NC4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:20.391Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-8ee9d58407d6c77532b674ca: done: 0.029908 sec","job_status":"done","queue_duration_s":0.309927,"scheduling_latency_s":0.309949,"redis_calls":10,"redis_duration_s":0.003726,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000437,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002252,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001037,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016886,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000988953,"duration_s":0.029908,"completed_at":"2026-06-14T15:02:20.730Z","load_balancing_strategy":"primary","db_duration_s":0.004494,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2095vm-test-run-nixbot-gitlab> gitlab # [  202.505306] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:20.738Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"0128daaf036bc89007a5a7a3","created_at":"2026-06-14T15:02:20.400Z","trace_propagation_headers":{"sentry-trace":"ce4cab28ec704bc6bdf3b972d8e39104-72552e6a23414af2","baggage":"sentry-trace_id=ce4cab28ec704bc6bdf3b972d8e39104,sentry-release=Unknown"},"correlation_id":"01KV3AFB4KYQPVX34QNX7T1NC4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:20.404Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-0128daaf036bc89007a5a7a3: start","job_status":"start","queue_duration_s":0.33395,"scheduling_latency_s":0.333987}2096vm-test-run-nixbot-gitlab> gitlab # [  202.511657] gitaly[1387]: time="2026-06-14T15:02:19.845Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFB4KYQPVX34QNX7T1NC4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T15:02:49.844" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:19.845" grpc.time_ms=0.544 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2097vm-test-run-nixbot-gitlab> gitlab # [  202.531338] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:20.768Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"0128daaf036bc89007a5a7a3","created_at":"2026-06-14T15:02:20.400Z","trace_propagation_headers":{"sentry-trace":"ce4cab28ec704bc6bdf3b972d8e39104-72552e6a23414af2","baggage":"sentry-trace_id=ce4cab28ec704bc6bdf3b972d8e39104,sentry-release=Unknown"},"correlation_id":"01KV3AFB4KYQPVX34QNX7T1NC4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:20.404Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-0128daaf036bc89007a5a7a3: done: 0.030426 sec","job_status":"done","queue_duration_s":0.33395,"scheduling_latency_s":0.333987,"redis_calls":13,"redis_duration_s":0.002791,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000418,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001304,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000629,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00044,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019502,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000381054,"duration_s":0.030426,"completed_at":"2026-06-14T15:02:20.768Z","load_balancing_strategy":"primary","db_duration_s":0.003671,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2098vm-test-run-nixbot-gitlab> gitlab # [  202.626254] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:20.770Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"d0512ef7136df4409c62786a","created_at":"2026-06-14T15:02:20.405Z","trace_propagation_headers":{"sentry-trace":"ce4cab28ec704bc6bdf3b972d8e39104-72552e6a23414af2","baggage":"sentry-trace_id=ce4cab28ec704bc6bdf3b972d8e39104,sentry-release=Unknown"},"correlation_id":"01KV3AFB4KYQPVX34QNX7T1NC4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:20.409Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-d0512ef7136df4409c62786a: start","job_status":"start","queue_duration_s":0.360765,"scheduling_latency_s":0.360785}2099vm-test-run-nixbot-gitlab> gitlab # [  202.628045] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:20.792Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"d0512ef7136df4409c62786a","created_at":"2026-06-14T15:02:20.405Z","trace_propagation_headers":{"sentry-trace":"ce4cab28ec704bc6bdf3b972d8e39104-72552e6a23414af2","baggage":"sentry-trace_id=ce4cab28ec704bc6bdf3b972d8e39104,sentry-release=Unknown"},"correlation_id":"01KV3AFB4KYQPVX34QNX7T1NC4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:20.409Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-d0512ef7136df4409c62786a: done: 0.022302 sec","job_status":"done","queue_duration_s":0.360765,"scheduling_latency_s":0.360785,"redis_calls":10,"redis_duration_s":0.0021520000000000003,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000441,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001304,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000407,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013563,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000337752,"duration_s":0.022302,"completed_at":"2026-06-14T15:02:20.792Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002806,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2100vm-test-run-nixbot-gitlab> gitlab # [  202.700100] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:20.794Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"6fc13a5ae8c1743729fec80e","created_at":"2026-06-14T15:02:20.410Z","trace_propagation_headers":{"sentry-trace":"ce4cab28ec704bc6bdf3b972d8e39104-72552e6a23414af2","baggage":"sentry-trace_id=ce4cab28ec704bc6bdf3b972d8e39104,sentry-release=Unknown"},"correlation_id":"01KV3AFB4KYQPVX34QNX7T1NC4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:20.414Z","job_size_bytes":19,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-6fc13a5ae8c1743729fec80e: start","job_status":"start","queue_duration_s":0.379395,"scheduling_latency_s":0.379416}2101vm-test-run-nixbot-gitlab> gitlab # [  202.725542] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:20.843Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"6fc13a5ae8c1743729fec80e","created_at":"2026-06-14T15:02:20.410Z","trace_propagation_headers":{"sentry-trace":"ce4cab28ec704bc6bdf3b972d8e39104-72552e6a23414af2","baggage":"sentry-trace_id=ce4cab28ec704bc6bdf3b972d8e39104,sentry-release=Unknown"},"correlation_id":"01KV3AFB4KYQPVX34QNX7T1NC4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:20.414Z","job_size_bytes":19,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-6fc13a5ae8c1743729fec80e: done: 0.04978 sec","job_status":"done","queue_duration_s":0.379395,"scheduling_latency_s":0.379416,"redis_calls":11,"redis_duration_s":0.002568,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000533,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001592,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000443,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.023,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017682,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000376305,"duration_s":0.04978,"completed_at":"2026-06-14T15:02:20.843Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.025799,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2102vm-test-run-nixbot-gitlab> gitlab # [  202.797260] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:20.845Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"d6d4eb412e4907f317102575","created_at":"2026-06-14T15:02:20.416Z","trace_propagation_headers":{"sentry-trace":"ce4cab28ec704bc6bdf3b972d8e39104-72552e6a23414af2","baggage":"sentry-trace_id=ce4cab28ec704bc6bdf3b972d8e39104,sentry-release=Unknown"},"correlation_id":"01KV3AFB4KYQPVX34QNX7T1NC4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:20.419Z","job_size_bytes":65,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-d6d4eb412e4907f317102575: start","job_status":"start","queue_duration_s":0.42524,"scheduling_latency_s":0.425261}2103vm-test-run-nixbot-gitlab> gitlab # [  202.867823] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:20.867Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"d6d4eb412e4907f317102575","created_at":"2026-06-14T15:02:20.416Z","trace_propagation_headers":{"sentry-trace":"ce4cab28ec704bc6bdf3b972d8e39104-72552e6a23414af2","baggage":"sentry-trace_id=ce4cab28ec704bc6bdf3b972d8e39104,sentry-release=Unknown"},"correlation_id":"01KV3AFB4KYQPVX34QNX7T1NC4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:20.419Z","job_size_bytes":65,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-d6d4eb412e4907f317102575: done: 0.021941 sec","job_status":"done","queue_duration_s":0.42524,"scheduling_latency_s":0.425261,"redis_calls":10,"redis_duration_s":0.002177,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000402,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001348,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000427,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013549,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000375746,"duration_s":0.021941,"completed_at":"2026-06-14T15:02:20.867Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002567,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2104vm-test-run-nixbot-gitlab> gitlab # [  202.936419] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:20.868Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"4174a2c377793fff2ac51abb","created_at":"2026-06-14T15:02:20.421Z","trace_propagation_headers":{"sentry-trace":"ce4cab28ec704bc6bdf3b972d8e39104-72552e6a23414af2","baggage":"sentry-trace_id=ce4cab28ec704bc6bdf3b972d8e39104,sentry-release=Unknown"},"correlation_id":"01KV3AFB4KYQPVX34QNX7T1NC4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:20.422Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-4174a2c377793fff2ac51abb: start","job_status":"start","queue_duration_s":0.445679,"scheduling_latency_s":0.445744}2105vm-test-run-nixbot-gitlab> gitlab # [  202.937715] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:20.890Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"4174a2c377793fff2ac51abb","created_at":"2026-06-14T15:02:20.421Z","trace_propagation_headers":{"sentry-trace":"ce4cab28ec704bc6bdf3b972d8e39104-72552e6a23414af2","baggage":"sentry-trace_id=ce4cab28ec704bc6bdf3b972d8e39104,sentry-release=Unknown"},"correlation_id":"01KV3AFB4KYQPVX34QNX7T1NC4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:20.422Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-4174a2c377793fff2ac51abb: done: 0.022432 sec","job_status":"done","queue_duration_s":0.445679,"scheduling_latency_s":0.445744,"redis_calls":7,"redis_duration_s":0.001635,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001145,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00049,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016057,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022432,"completed_at":"2026-06-14T15:02:20.890Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002564,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2106vm-test-run-nixbot-gitlab> gitlab # [  203.045935] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:20.892Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"119aa621882944e0d7188c65","created_at":"2026-06-14T15:02:20.450Z","trace_propagation_headers":{"sentry-trace":"ce4cab28ec704bc6bdf3b972d8e39104-72552e6a23414af2","baggage":"sentry-trace_id=ce4cab28ec704bc6bdf3b972d8e39104,sentry-release=Unknown"},"correlation_id":"01KV3AFB4KYQPVX34QNX7T1NC4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:20.454Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-119aa621882944e0d7188c65: start","job_status":"start","queue_duration_s":0.438149,"scheduling_latency_s":0.438171}2107vm-test-run-nixbot-gitlab> gitlab # [  203.047275] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:20.923Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"119aa621882944e0d7188c65","created_at":"2026-06-14T15:02:20.450Z","trace_propagation_headers":{"sentry-trace":"ce4cab28ec704bc6bdf3b972d8e39104-72552e6a23414af2","baggage":"sentry-trace_id=ce4cab28ec704bc6bdf3b972d8e39104,sentry-release=Unknown"},"correlation_id":"01KV3AFB4KYQPVX34QNX7T1NC4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:20.454Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-119aa621882944e0d7188c65: done: 0.030785 sec","job_status":"done","queue_duration_s":0.438149,"scheduling_latency_s":0.438171,"redis_calls":10,"redis_duration_s":0.0027879999999999997,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000667,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001633,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000488,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018078,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000350603,"duration_s":0.030785,"completed_at":"2026-06-14T15:02:20.923Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006957,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2108vm-test-run-nixbot-gitlab> gitlab # [  203.119341] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:20.924Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"ebe88912253215013b8e6095","created_at":"2026-06-14T15:02:20.454Z","trace_propagation_headers":{"sentry-trace":"ce4cab28ec704bc6bdf3b972d8e39104-72552e6a23414af2","baggage":"sentry-trace_id=ce4cab28ec704bc6bdf3b972d8e39104,sentry-release=Unknown"},"correlation_id":"01KV3AFB4KYQPVX34QNX7T1NC4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:20.457Z","job_size_bytes":6,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-ebe88912253215013b8e6095: start","job_status":"start","queue_duration_s":0.466915,"scheduling_latency_s":0.466936}2109vm-test-run-nixbot-gitlab> gitlab # [  203.120378] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:20.942Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"ebe88912253215013b8e6095","created_at":"2026-06-14T15:02:20.454Z","trace_propagation_headers":{"sentry-trace":"ce4cab28ec704bc6bdf3b972d8e39104-72552e6a23414af2","baggage":"sentry-trace_id=ce4cab28ec704bc6bdf3b972d8e39104,sentry-release=Unknown"},"correlation_id":"01KV3AFB4KYQPVX34QNX7T1NC4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:20.457Z","job_size_bytes":6,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-ebe88912253215013b8e6095: done: 0.018401 sec","job_status":"done","queue_duration_s":0.466915,"scheduling_latency_s":0.466936,"redis_calls":10,"redis_duration_s":0.002152,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000427,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001277,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000448,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011014,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000381333,"duration_s":0.018401,"completed_at":"2026-06-14T15:02:20.942Z","load_balancing_strategy":"primary","db_duration_s":0.001588,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2110vm-test-run-nixbot-gitlab> gitlab # [  203.203178] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:20.944Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"e0d36c1c788b2db14f6bf4a6","created_at":"2026-06-14T15:02:20.484Z","trace_propagation_headers":{"sentry-trace":"ce4cab28ec704bc6bdf3b972d8e39104-72552e6a23414af2","baggage":"sentry-trace_id=ce4cab28ec704bc6bdf3b972d8e39104,sentry-release=Unknown"},"correlation_id":"01KV3AFB4KYQPVX34QNX7T1NC4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:20.485Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-e0d36c1c788b2db14f6bf4a6: start","job_status":"start","queue_duration_s":0.458805,"scheduling_latency_s":0.458826}2111vm-test-run-nixbot-gitlab> gitlab # [  203.204217] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:20.959Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"e0d36c1c788b2db14f6bf4a6","created_at":"2026-06-14T15:02:20.484Z","trace_propagation_headers":{"sentry-trace":"ce4cab28ec704bc6bdf3b972d8e39104-72552e6a23414af2","baggage":"sentry-trace_id=ce4cab28ec704bc6bdf3b972d8e39104,sentry-release=Unknown"},"correlation_id":"01KV3AFB4KYQPVX34QNX7T1NC4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:20.485Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-e0d36c1c788b2db14f6bf4a6: done: 0.015469 sec","job_status":"done","queue_duration_s":0.458805,"scheduling_latency_s":0.458826,"redis_calls":8,"redis_duration_s":0.0018050000000000002,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001278,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000527,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010063,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015469,"completed_at":"2026-06-14T15:02:20.959Z","load_balancing_strategy":"primary","db_duration_s":0.001322,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2112vm-test-run-nixbot-gitlab> gitlab # [  203.266400] gitaly[1387]: time="2026-06-14T15:02:19.854Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KV3AFCW0TMEDD58CW2Y1WZZZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-14T15:02:29.127" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T15:02:19.847" grpc.time_ms=6.912 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2113vm-test-run-nixbot-gitlab> gitlab # [  203.342993] gitaly[1387]: time="2026-06-14T15:02:20.169Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV3AFCJ9XJJ8HRSH21SEYHPD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-14T15:02:50.167" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:20.168" grpc.time_ms=1.791 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2114vm-test-run-nixbot-gitlab> gitlab # [  203.344281] gitaly[1387]: time="2026-06-14T15:02:20.194Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFCJ9XJJ8HRSH21SEYHPD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T15:02:50.193" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:20.193" grpc.time_ms=0.664 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2115vm-test-run-nixbot-gitlab> gitlab # [  203.345202] gitaly[1387]: time="2026-06-14T15:02:20.215Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFCJ9XJJ8HRSH21SEYHPD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T15:02:50.214" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:20.214" grpc.time_ms=0.479 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2116vm-test-run-nixbot-gitlab> gitlab # [  203.346005] gitaly[1387]: time="2026-06-14T15:02:20.311Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV3AFB4KYQPVX34QNX7T1NC4 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T15:02:20.311" grpc.time_ms=0.456 pid=1387 span.kind=server system=grpc2117vm-test-run-nixbot-gitlab> gitlab # [  203.361289] gitaly[1387]: time="2026-06-14T15:02:20.333Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV3AFB4KYQPVX34QNX7T1NC4 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T15:02:20.333" grpc.time_ms=0.471 pid=1387 span.kind=server system=grpc2118vm-test-run-nixbot-gitlab> gitlab # [  203.376903] gitaly[1387]: time="2026-06-14T15:02:20.355Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV3AFB4KYQPVX34QNX7T1NC4 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T15:02:20.354" grpc.time_ms=0.322 pid=1387 span.kind=server system=grpc2119vm-test-run-nixbot-gitlab> gitlab # [  203.392379] gitaly[1387]: time="2026-06-14T15:02:20.366Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFB4KYQPVX34QNX7T1NC4 diskcache=c17d6e6d-3033-47a9-b0ed-09173701252c grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-14T15:02:50.286" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-14T15:02:20.287" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2120vm-test-run-nixbot-gitlab> gitlab # [  203.408050] gitaly[1387]: time="2026-06-14T15:02:20.366Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=79 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7080 command.oublock=0 command.real_time_ms=70 command.spawn_token_fork_ms=0 command.system_time_ms=50 command.user_time_ms=29 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFB4KYQPVX34QNX7T1NC4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-14T15:02:50.286" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T15:02:20.287" grpc.time_ms=79.442 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2121vm-test-run-nixbot-gitlab> gitlab # [  203.493164] gitaly[1387]: time="2026-06-14T15:02:20.428Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV3AFB4KYQPVX34QNX7T1NC4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-14T15:02:50.427" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:20.427" grpc.time_ms=1.053 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2122vm-test-run-nixbot-gitlab> gitlab # [  203.494433] gitaly[1387]: time="2026-06-14T15:02:20.449Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFB4KYQPVX34QNX7T1NC4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T15:02:50.448" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:20.448" grpc.time_ms=0.571 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2123vm-test-run-nixbot-gitlab> gitlab # [  203.495307] gitaly[1387]: time="2026-06-14T15:02:20.477Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFB4KYQPVX34QNX7T1NC4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T15:02:50.476" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:20.477" grpc.time_ms=0.568 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2124vm-test-run-nixbot-gitlab> gitlab # [  203.565848] nixbot[3073]: {"ts": "2026-06-14T15:02:22.827Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/5bc925784851d57384be462c5a4edebddba61b3c \"HTTP/1.1 201 Created\""}2125vm-test-run-nixbot-gitlab> gitlab # [  203.575581] nixbot[3073]: INFO:     ::ffff:127.0.0.1:48676 - "GET /api/repos/gitlab/root/test-flake/builds?commit=2ee4ce11810ac90f03a8181f7104629d7f1fa32e HTTP/1.1" 200 OK2126vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=2ee4ce11810ac90f03a8181f7104629d7f1fa32e', in 2.85 seconds)2127vm-test-run-nixbot-gitlab> [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '2ee4ce11810ac90f03a8181f7104629d7f1fa32e', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'evaluating', 'eval_warnings': None, 'error': None, 'created_at': '2026-06-14T15:02:19.415409Z', 'started_at': '2026-06-14T15:02:19.861019Z', 'finished_at': None}]2128vm-test-run-nixbot-gitlab> (finished: subtest: push delivers the webhook, in 6.32 seconds)2129vm-test-run-nixbot-gitlab> subtest: eval and build post commit statuses2130vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/2ee4ce11810ac90f03a8181f7104629d7f1fa32e/statuses2131vm-test-run-nixbot-gitlab> gitlab # [  203.626357] nix-daemon[3369]: accepted connection from pid 3462, user nixbot2132vm-test-run-nixbot-gitlab> gitlab # [  203.628666] nix-daemon[3369]: accepted connection from pid 3463, user nixbot2133vm-test-run-nixbot-gitlab> gitlab # [  203.673470] gitaly[1387]: time="2026-06-14T15:02:22.935Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFFXAVHND1FGZDNDD8RVW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T15:02:52.833" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:22.934" grpc.time_ms=0.662 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2134vm-test-run-nixbot-gitlab> gitlab # [  203.699824] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:22 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/2ee4ce11810ac90f03a8181f7104629d7f1fa32e/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 702135vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/2ee4ce11810ac90f03a8181f7104629d7f1fa32e/statuses, in 0.10 seconds)2136vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '2ee4ce11810ac90f03a8181f7104629d7f1fa32e', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-14T15:02:19.830Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2137vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/2ee4ce11810ac90f03a8181f7104629d7f1fa32e/statuses2138vm-test-run-nixbot-gitlab> gitlab # [  204.824298] gitaly[1387]: time="2026-06-14T15:02:24.085Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFH08WACRZZJN3DG6PN9Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T15:02:54.083" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:24.084" grpc.time_ms=1.023 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2139vm-test-run-nixbot-gitlab> gitlab # [  204.873751] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:24 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/2ee4ce11810ac90f03a8181f7104629d7f1fa32e/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1032140vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/2ee4ce11810ac90f03a8181f7104629d7f1fa32e/statuses, in 0.17 seconds)2141vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '2ee4ce11810ac90f03a8181f7104629d7f1fa32e', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-14T15:02:19.830Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2142vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/2ee4ce11810ac90f03a8181f7104629d7f1fa32e/statuses2143vm-test-run-nixbot-gitlab> gitlab # [  205.985328] gitaly[1387]: time="2026-06-14T15:02:25.246Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFJ4YMZ6RRAA0BECPQY1C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T15:02:55.144" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:25.245" grpc.time_ms=0.95 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2144vm-test-run-nixbot-gitlab> gitlab # [  206.057611] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:25 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/2ee4ce11810ac90f03a8181f7104629d7f1fa32e/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1362145vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/2ee4ce11810ac90f03a8181f7104629d7f1fa32e/statuses, in 0.19 seconds)2146vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '2ee4ce11810ac90f03a8181f7104629d7f1fa32e', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-14T15:02:19.830Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2147vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/2ee4ce11810ac90f03a8181f7104629d7f1fa32e/statuses2148vm-test-run-nixbot-gitlab> gitlab # [  207.189773] gitaly[1387]: time="2026-06-14T15:02:26.451Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFKA4JAF5W0N0BJ4NVWNR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T15:02:56.348" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:26.449" grpc.time_ms=1.322 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2149vm-test-run-nixbot-gitlab> gitlab # [  207.223776] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:26 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/2ee4ce11810ac90f03a8181f7104629d7f1fa32e/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1122150vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/2ee4ce11810ac90f03a8181f7104629d7f1fa32e/statuses, in 0.18 seconds)2151vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '2ee4ce11810ac90f03a8181f7104629d7f1fa32e', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-14T15:02:19.830Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2152vm-test-run-nixbot-gitlab> gitlab # [  208.240129] nix-daemon[3369]: accepted connection from pid 3495, user nixbot2153vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/2ee4ce11810ac90f03a8181f7104629d7f1fa32e/statuses2154vm-test-run-nixbot-gitlab> gitlab # [  208.302355] gitaly[1387]: time="2026-06-14T15:02:27.564Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFMCNXRMF99GDJ4PE6V54 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T15:02:57.462" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:27.563" grpc.time_ms=0.548 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2155vm-test-run-nixbot-gitlab> gitlab # [  208.351778] gitaly[1387]: time="2026-06-14T15:02:27.614Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFMEK9K9EC2DPB41VF67Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T15:02:57.612" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:27.613" grpc.time_ms=0.493 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2156vm-test-run-nixbot-gitlab> gitlab # [  208.373569] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:27 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/2ee4ce11810ac90f03a8181f7104629d7f1fa32e/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 942157vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/2ee4ce11810ac90f03a8181f7104629d7f1fa32e/statuses, in 0.13 seconds)2158vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '2ee4ce11810ac90f03a8181f7104629d7f1fa32e', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-14T15:02:19.830Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2159vm-test-run-nixbot-gitlab> gitlab # [  208.403327] gitaly[1387]: time="2026-06-14T15:02:27.665Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV3AFMCNXRMF99GDJ4PE6V54 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-14T15:02:37.116" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T15:02:27.656" grpc.time_ms=8.576 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2160vm-test-run-nixbot-gitlab> gitlab # [  208.436679] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:27.698Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"826d3ad5edc04e0523a46828","created_at":"2026-06-14T15:02:27.693Z","trace_propagation_headers":{"sentry-trace":"632aa37fa1ec492a975ce4e583b089d4-7b0e03fb6b594592","baggage":"sentry-trace_id=632aa37fa1ec492a975ce4e583b089d4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV3AFMCNXRMF99GDJ4PE6V54","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:27.696Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-826d3ad5edc04e0523a46828: start","job_status":"start","queue_duration_s":0.001766,"scheduling_latency_s":0.001805}2161vm-test-run-nixbot-gitlab> gitlab # [  208.462488] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:27 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/2ee4ce11810ac90f03a8181f7104629d7f1fa32e HTTP/1.1" 201 669 "" "python-httpx/0.28.1" 2422162vm-test-run-nixbot-gitlab> gitlab # [  208.468171] nixbot[3073]: {"ts": "2026-06-14T15:02:27.721Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/2ee4ce11810ac90f03a8181f7104629d7f1fa32e \"HTTP/1.1 201 Created\""}2163vm-test-run-nixbot-gitlab> gitlab # [  208.516103] gitaly[1387]: time="2026-06-14T15:02:27.778Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFMMFHNRWF757BXSHMXRR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T15:02:57.776" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:27.777" grpc.time_ms=0.736 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2164vm-test-run-nixbot-gitlab> gitlab # [  208.545976] gitaly[1387]: time="2026-06-14T15:02:27.808Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV3AFMMFHNRWF757BXSHMXRR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-14T15:02:37.081" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T15:02:27.802" grpc.time_ms=6.042 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2165vm-test-run-nixbot-gitlab> gitlab # [  208.591687] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:27 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/2ee4ce11810ac90f03a8181f7104629d7f1fa32e HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1262166vm-test-run-nixbot-gitlab> gitlab # [  208.597054] nixbot[3073]: {"ts": "2026-06-14T15:02:27.859Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/2ee4ce11810ac90f03a8181f7104629d7f1fa32e \"HTTP/1.1 201 Created\""}2167vm-test-run-nixbot-gitlab> gitlab # [  208.646506] gitaly[1387]: time="2026-06-14T15:02:27.908Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV3AFMCNXRMF99GDJ4PE6V54 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T15:02:27.908" grpc.time_ms=0.389 pid=1387 span.kind=server system=grpc2168vm-test-run-nixbot-gitlab> gitlab # [  208.691240] gitaly[1387]: time="2026-06-14T15:02:27.953Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV3AFMCNXRMF99GDJ4PE6V54 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T15:02:27.952" grpc.time_ms=0.623 pid=1387 span.kind=server system=grpc2169vm-test-run-nixbot-gitlab> gitlab # [  208.730887] gitaly[1387]: time="2026-06-14T15:02:27.993Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV3AFMCNXRMF99GDJ4PE6V54 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T15:02:27.992" grpc.time_ms=0.477 pid=1387 span.kind=server system=grpc2170vm-test-run-nixbot-gitlab> gitlab # [  208.758399] gitaly[1387]: time="2026-06-14T15:02:28.020Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFMCNXRMF99GDJ4PE6V54 diskcache=40a93859-5b5f-4584-a78c-40a3215b249a grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-14T15:02:57.881" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-14T15:02:27.881" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2171vm-test-run-nixbot-gitlab> gitlab # [  208.758940] gitaly[1387]: time="2026-06-14T15:02:28.021Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=118 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7304 command.oublock=0 command.real_time_ms=131 command.spawn_token_fork_ms=0 command.system_time_ms=77 command.user_time_ms=41 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFMCNXRMF99GDJ4PE6V54 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-14T15:02:57.881" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T15:02:27.881" grpc.time_ms=138.979 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2172vm-test-run-nixbot-gitlab> gitlab # [  208.796620] gitaly[1387]: time="2026-06-14T15:02:28.058Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV3AFMCNXRMF99GDJ4PE6V54 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-14T15:02:58.057" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:28.057" grpc.time_ms=0.857 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2173vm-test-run-nixbot-gitlab> gitlab # [  208.820454] gitaly[1387]: time="2026-06-14T15:02:28.082Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFMCNXRMF99GDJ4PE6V54 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T15:02:58.080" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:28.081" grpc.time_ms=0.767 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2174vm-test-run-nixbot-gitlab> gitlab # [  208.856313] gitaly[1387]: time="2026-06-14T15:02:28.118Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFMCNXRMF99GDJ4PE6V54 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T15:02:58.116" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:28.116" grpc.time_ms=1.265 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2175vm-test-run-nixbot-gitlab> gitlab # [  208.876402] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:28.134Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"3bfee2c6039903d4ed560eb7","created_at":"2026-06-14T15:02:28.131Z","trace_propagation_headers":{"sentry-trace":"632aa37fa1ec492a975ce4e583b089d4-84bc648976064805","baggage":"sentry-trace_id=632aa37fa1ec492a975ce4e583b089d4,sentry-release=Unknown"},"correlation_id":"01KV3AFMCNXRMF99GDJ4PE6V54","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"826d3ad5edc04e0523a46828","job_size_bytes":3,"pid":2097,"job_status":"deduplicated","message":"PipelineProcessWorker JID-3bfee2c6039903d4ed560eb7: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}2176vm-test-run-nixbot-gitlab> gitlab # [  208.876899] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:28.137Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"826d3ad5edc04e0523a46828","created_at":"2026-06-14T15:02:27.693Z","trace_propagation_headers":{"sentry-trace":"632aa37fa1ec492a975ce4e583b089d4-7b0e03fb6b594592","baggage":"sentry-trace_id=632aa37fa1ec492a975ce4e583b089d4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV3AFMCNXRMF99GDJ4PE6V54","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:27.696Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2097,"job_status":"rescheduled","message":"PipelineProcessWorker JID-826d3ad5edc04e0523a46828: rescheduled"}2177vm-test-run-nixbot-gitlab> gitlab # [  208.887704] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:28.149Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"826d3ad5edc04e0523a46828","created_at":"2026-06-14T15:02:27.693Z","trace_propagation_headers":{"sentry-trace":"632aa37fa1ec492a975ce4e583b089d4-7b0e03fb6b594592","baggage":"sentry-trace_id=632aa37fa1ec492a975ce4e583b089d4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV3AFMCNXRMF99GDJ4PE6V54","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:27.696Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-826d3ad5edc04e0523a46828: done: 0.451123 sec","job_status":"done","queue_duration_s":0.001766,"scheduling_latency_s":0.001805,"gitaly_calls":4,"gitaly_duration_s":0.151468,"redis_calls":93,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.02636,"redis_read_bytes":262,"redis_write_bytes":23391,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001048,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":21,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.0036,"redis_cache_read_bytes":39,"redis_cache_write_bytes":1840,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001294,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":712,"redis_queues_calls":25,"redis_queues_duration_s":0.005373,"redis_queues_read_bytes":28,"redis_queues_write_bytes":14472,"redis_queues_metadata_calls":23,"redis_queues_metadata_duration_s":0.008977,"redis_queues_metadata_read_bytes":112,"redis_queues_metadata_write_bytes":3754,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.00025,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001267,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.003503,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":38,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":38,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.039,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":23,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.116,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.123,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.02,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.12868,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.407060395,"duration_s":0.451123,"completed_at":"2026-06-14T15:02:28.149Z","load_balancing_strategy":"primary","db_duration_s":0.15369,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2178vm-test-run-nixbot-gitlab> gitlab # [  208.973772] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:28.153Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"1080c0a72c6753b5321a5e4d","created_at":"2026-06-14T15:02:27.874Z","trace_propagation_headers":{"sentry-trace":"632aa37fa1ec492a975ce4e583b089d4-84bc648976064805","baggage":"sentry-trace_id=632aa37fa1ec492a975ce4e583b089d4,sentry-release=Unknown"},"correlation_id":"01KV3AFMCNXRMF99GDJ4PE6V54","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:27.876Z","job_size_bytes":23,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-1080c0a72c6753b5321a5e4d: start","job_status":"start","queue_duration_s":0.27612,"scheduling_latency_s":0.276171}2179vm-test-run-nixbot-gitlab> gitlab # [  208.975538] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:28.198Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"1080c0a72c6753b5321a5e4d","created_at":"2026-06-14T15:02:27.874Z","trace_propagation_headers":{"sentry-trace":"632aa37fa1ec492a975ce4e583b089d4-84bc648976064805","baggage":"sentry-trace_id=632aa37fa1ec492a975ce4e583b089d4,sentry-release=Unknown"},"correlation_id":"01KV3AFMCNXRMF99GDJ4PE6V54","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:27.876Z","job_size_bytes":23,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-1080c0a72c6753b5321a5e4d: done: 0.045856 sec","job_status":"done","queue_duration_s":0.27612,"scheduling_latency_s":0.276171,"redis_calls":7,"redis_duration_s":0.003594,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002627,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000967,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028208,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.045856,"completed_at":"2026-06-14T15:02:28.198Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008836,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2180vm-test-run-nixbot-gitlab> gitlab # [  209.062538] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:28.200Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"75259dc466a0c03ff67625b3","created_at":"2026-06-14T15:02:28.025Z","trace_propagation_headers":{"sentry-trace":"632aa37fa1ec492a975ce4e583b089d4-84bc648976064805","baggage":"sentry-trace_id=632aa37fa1ec492a975ce4e583b089d4,sentry-release=Unknown"},"correlation_id":"01KV3AFMCNXRMF99GDJ4PE6V54","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:28.028Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-75259dc466a0c03ff67625b3: start","job_status":"start","queue_duration_s":0.172466,"scheduling_latency_s":0.172492}2181vm-test-run-nixbot-gitlab> gitlab # [  209.064277] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:28.232Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"75259dc466a0c03ff67625b3","created_at":"2026-06-14T15:02:28.025Z","trace_propagation_headers":{"sentry-trace":"632aa37fa1ec492a975ce4e583b089d4-84bc648976064805","baggage":"sentry-trace_id=632aa37fa1ec492a975ce4e583b089d4,sentry-release=Unknown"},"correlation_id":"01KV3AFMCNXRMF99GDJ4PE6V54","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:28.028Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-75259dc466a0c03ff67625b3: done: 0.031677 sec","job_status":"done","queue_duration_s":0.172466,"scheduling_latency_s":0.172492,"redis_calls":7,"redis_duration_s":0.003001,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002194,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000807,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02051,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031677,"completed_at":"2026-06-14T15:02:28.232Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003802,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2182vm-test-run-nixbot-gitlab> gitlab # [  209.174293] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:28.235Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"eb76e3b6122ff779a3c4f041","created_at":"2026-06-14T15:02:28.035Z","trace_propagation_headers":{"sentry-trace":"632aa37fa1ec492a975ce4e583b089d4-84bc648976064805","baggage":"sentry-trace_id=632aa37fa1ec492a975ce4e583b089d4,sentry-release=Unknown"},"correlation_id":"01KV3AFMCNXRMF99GDJ4PE6V54","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:28.036Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-eb76e3b6122ff779a3c4f041: start","job_status":"start","queue_duration_s":0.1985,"scheduling_latency_s":0.198556}2183vm-test-run-nixbot-gitlab> gitlab # [  209.175944] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:28.298Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"eb76e3b6122ff779a3c4f041","created_at":"2026-06-14T15:02:28.035Z","trace_propagation_headers":{"sentry-trace":"632aa37fa1ec492a975ce4e583b089d4-84bc648976064805","baggage":"sentry-trace_id=632aa37fa1ec492a975ce4e583b089d4,sentry-release=Unknown"},"correlation_id":"01KV3AFMCNXRMF99GDJ4PE6V54","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:28.036Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-eb76e3b6122ff779a3c4f041: done: 0.063453 sec","job_status":"done","queue_duration_s":0.1985,"scheduling_latency_s":0.198556,"redis_calls":7,"redis_duration_s":0.00279,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00204,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00075,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.026,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026828,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.063453,"completed_at":"2026-06-14T15:02:28.298Z","load_balancing_strategy":"primary","db_duration_s":0.026419,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2184vm-test-run-nixbot-gitlab> gitlab # [  209.291147] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:28.301Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"06c9e38986ed5b2bc1db9d5f","created_at":"2026-06-14T15:02:28.043Z","trace_propagation_headers":{"sentry-trace":"632aa37fa1ec492a975ce4e583b089d4-84bc648976064805","baggage":"sentry-trace_id=632aa37fa1ec492a975ce4e583b089d4,sentry-release=Unknown"},"correlation_id":"01KV3AFMCNXRMF99GDJ4PE6V54","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:28.044Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-06c9e38986ed5b2bc1db9d5f: start","job_status":"start","queue_duration_s":0.25649,"scheduling_latency_s":0.256552}2185vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/2ee4ce11810ac90f03a8181f7104629d7f1fa32e/statuses2186vm-test-run-nixbot-gitlab> gitlab # [  209.292750] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:28.329Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"06c9e38986ed5b2bc1db9d5f","created_at":"2026-06-14T15:02:28.043Z","trace_propagation_headers":{"sentry-trace":"632aa37fa1ec492a975ce4e583b089d4-84bc648976064805","baggage":"sentry-trace_id=632aa37fa1ec492a975ce4e583b089d4,sentry-release=Unknown"},"correlation_id":"01KV3AFMCNXRMF99GDJ4PE6V54","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:28.044Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-06c9e38986ed5b2bc1db9d5f: done: 0.028087 sec","job_status":"done","queue_duration_s":0.25649,"scheduling_latency_s":0.256552,"redis_calls":10,"redis_duration_s":0.002437,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001316,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000618,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000503,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017482,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028087,"completed_at":"2026-06-14T15:02:28.328Z","load_balancing_strategy":"primary","db_duration_s":0.003902,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2187vm-test-run-nixbot-gitlab> gitlab # [  209.383175] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:28.330Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"395db4527f39e59f1dd38035","created_at":"2026-06-14T15:02:28.045Z","trace_propagation_headers":{"sentry-trace":"632aa37fa1ec492a975ce4e583b089d4-84bc648976064805","baggage":"sentry-trace_id=632aa37fa1ec492a975ce4e583b089d4,sentry-release=Unknown"},"correlation_id":"01KV3AFMCNXRMF99GDJ4PE6V54","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:28.046Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-395db4527f39e59f1dd38035: start","job_status":"start","queue_duration_s":0.28405,"scheduling_latency_s":0.284073}2188vm-test-run-nixbot-gitlab> gitlab # [  209.384384] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:28.347Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"395db4527f39e59f1dd38035","created_at":"2026-06-14T15:02:28.045Z","trace_propagation_headers":{"sentry-trace":"632aa37fa1ec492a975ce4e583b089d4-84bc648976064805","baggage":"sentry-trace_id=632aa37fa1ec492a975ce4e583b089d4,sentry-release=Unknown"},"correlation_id":"01KV3AFMCNXRMF99GDJ4PE6V54","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:28.046Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-395db4527f39e59f1dd38035: done: 0.01713 sec","job_status":"done","queue_duration_s":0.28405,"scheduling_latency_s":0.284073,"redis_calls":7,"redis_duration_s":0.001503,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001086,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000417,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010478,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01713,"completed_at":"2026-06-14T15:02:28.347Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002812,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2189vm-test-run-nixbot-gitlab> gitlab # [  209.457795] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:28.348Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"99bda6911a3e0d3a6c1fed7f","created_at":"2026-06-14T15:02:28.046Z","trace_propagation_headers":{"sentry-trace":"632aa37fa1ec492a975ce4e583b089d4-84bc648976064805","baggage":"sentry-trace_id=632aa37fa1ec492a975ce4e583b089d4,sentry-release=Unknown"},"correlation_id":"01KV3AFMCNXRMF99GDJ4PE6V54","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:28.048Z","job_size_bytes":19,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-99bda6911a3e0d3a6c1fed7f: start","job_status":"start","queue_duration_s":0.300074,"scheduling_latency_s":0.300095}2190vm-test-run-nixbot-gitlab> gitlab # [  209.459468] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:28.366Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"99bda6911a3e0d3a6c1fed7f","created_at":"2026-06-14T15:02:28.046Z","trace_propagation_headers":{"sentry-trace":"632aa37fa1ec492a975ce4e583b089d4-84bc648976064805","baggage":"sentry-trace_id=632aa37fa1ec492a975ce4e583b089d4,sentry-release=Unknown"},"correlation_id":"01KV3AFMCNXRMF99GDJ4PE6V54","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:28.048Z","job_size_bytes":19,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-99bda6911a3e0d3a6c1fed7f: done: 0.017832 sec","job_status":"done","queue_duration_s":0.300074,"scheduling_latency_s":0.300095,"redis_calls":8,"redis_duration_s":0.0017130000000000001,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001296,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000417,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011682,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017832,"completed_at":"2026-06-14T15:02:28.366Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002185,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2191vm-test-run-nixbot-gitlab> gitlab # [  209.505590] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:28.367Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"31b1b38856958b97738280f1","created_at":"2026-06-14T15:02:28.049Z","trace_propagation_headers":{"sentry-trace":"632aa37fa1ec492a975ce4e583b089d4-84bc648976064805","baggage":"sentry-trace_id=632aa37fa1ec492a975ce4e583b089d4,sentry-release=Unknown"},"correlation_id":"01KV3AFMCNXRMF99GDJ4PE6V54","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:28.050Z","job_size_bytes":66,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-31b1b38856958b97738280f1: start","job_status":"start","queue_duration_s":0.317195,"scheduling_latency_s":0.317218}2192vm-test-run-nixbot-gitlab> gitlab # [  209.583272] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:28.382Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"31b1b38856958b97738280f1","created_at":"2026-06-14T15:02:28.049Z","trace_propagation_headers":{"sentry-trace":"632aa37fa1ec492a975ce4e583b089d4-84bc648976064805","baggage":"sentry-trace_id=632aa37fa1ec492a975ce4e583b089d4,sentry-release=Unknown"},"correlation_id":"01KV3AFMCNXRMF99GDJ4PE6V54","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:28.050Z","job_size_bytes":66,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-31b1b38856958b97738280f1: done: 0.014605 sec","job_status":"done","queue_duration_s":0.317195,"scheduling_latency_s":0.317218,"redis_calls":7,"redis_duration_s":0.0015680000000000002,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001146,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000422,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009646,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014605,"completed_at":"2026-06-14T15:02:28.382Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000971,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2193vm-test-run-nixbot-gitlab> gitlab # [  209.602070] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:28.383Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"533c67578be18312cd327614","created_at":"2026-06-14T15:02:28.051Z","trace_propagation_headers":{"sentry-trace":"632aa37fa1ec492a975ce4e583b089d4-84bc648976064805","baggage":"sentry-trace_id=632aa37fa1ec492a975ce4e583b089d4,sentry-release=Unknown"},"correlation_id":"01KV3AFMCNXRMF99GDJ4PE6V54","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:28.052Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-533c67578be18312cd327614: start","job_status":"start","queue_duration_s":0.331082,"scheduling_latency_s":0.331103}2194vm-test-run-nixbot-gitlab> gitlab # [  209.672943] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:28.405Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"533c67578be18312cd327614","created_at":"2026-06-14T15:02:28.051Z","trace_propagation_headers":{"sentry-trace":"632aa37fa1ec492a975ce4e583b089d4-84bc648976064805","baggage":"sentry-trace_id=632aa37fa1ec492a975ce4e583b089d4,sentry-release=Unknown"},"correlation_id":"01KV3AFMCNXRMF99GDJ4PE6V54","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:28.052Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-533c67578be18312cd327614: done: 0.021891 sec","job_status":"done","queue_duration_s":0.331082,"scheduling_latency_s":0.331103,"redis_calls":7,"redis_duration_s":0.001603,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001182,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000421,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015544,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021891,"completed_at":"2026-06-14T15:02:28.405Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002524,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2195vm-test-run-nixbot-gitlab> gitlab # [  209.693849] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:28.406Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"15e6279df97b6374fcf0bf71","created_at":"2026-06-14T15:02:28.088Z","trace_propagation_headers":{"sentry-trace":"632aa37fa1ec492a975ce4e583b089d4-84bc648976064805","baggage":"sentry-trace_id=632aa37fa1ec492a975ce4e583b089d4,sentry-release=Unknown"},"correlation_id":"01KV3AFMCNXRMF99GDJ4PE6V54","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:28.089Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-15e6279df97b6374fcf0bf71: start","job_status":"start","queue_duration_s":0.317573,"scheduling_latency_s":0.317594}2196vm-test-run-nixbot-gitlab> gitlab # [  209.720250] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:28.424Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"15e6279df97b6374fcf0bf71","created_at":"2026-06-14T15:02:28.088Z","trace_propagation_headers":{"sentry-trace":"632aa37fa1ec492a975ce4e583b089d4-84bc648976064805","baggage":"sentry-trace_id=632aa37fa1ec492a975ce4e583b089d4,sentry-release=Unknown"},"correlation_id":"01KV3AFMCNXRMF99GDJ4PE6V54","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:28.089Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-15e6279df97b6374fcf0bf71: done: 0.017252 sec","job_status":"done","queue_duration_s":0.317573,"scheduling_latency_s":0.317594,"redis_calls":7,"redis_duration_s":0.001524,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001116,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000408,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010553,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017252,"completed_at":"2026-06-14T15:02:28.424Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002893,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2197vm-test-run-nixbot-gitlab> gitlab # [  209.788859] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:28.425Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"f1c984861f5047844032fbd9","created_at":"2026-06-14T15:02:28.089Z","trace_propagation_headers":{"sentry-trace":"632aa37fa1ec492a975ce4e583b089d4-84bc648976064805","baggage":"sentry-trace_id=632aa37fa1ec492a975ce4e583b089d4,sentry-release=Unknown"},"correlation_id":"01KV3AFMCNXRMF99GDJ4PE6V54","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:28.091Z","job_size_bytes":6,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-f1c984861f5047844032fbd9: start","job_status":"start","queue_duration_s":0.334221,"scheduling_latency_s":0.334242}2198vm-test-run-nixbot-gitlab> gitlab # [  209.859930] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:28.437Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"f1c984861f5047844032fbd9","created_at":"2026-06-14T15:02:28.089Z","trace_propagation_headers":{"sentry-trace":"632aa37fa1ec492a975ce4e583b089d4-84bc648976064805","baggage":"sentry-trace_id=632aa37fa1ec492a975ce4e583b089d4,sentry-release=Unknown"},"correlation_id":"01KV3AFMCNXRMF99GDJ4PE6V54","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:28.091Z","job_size_bytes":6,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-f1c984861f5047844032fbd9: done: 0.0126 sec","job_status":"done","queue_duration_s":0.334221,"scheduling_latency_s":0.334242,"redis_calls":7,"redis_duration_s":0.002013,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001542,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000471,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007823,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0126,"completed_at":"2026-06-14T15:02:28.437Z","load_balancing_strategy":"primary","db_duration_s":0.000401,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2199vm-test-run-nixbot-gitlab> gitlab # [  209.877404] gitaly[1387]: time="2026-06-14T15:02:28.584Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV3AFMCNXRMF99GDJ4PE6V54 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-14T15:02:58.583" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:28.583" grpc.time_ms=0.701 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2200vm-test-run-nixbot-gitlab> gitlab # [  209.959372] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:28 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/2ee4ce11810ac90f03a8181f7104629d7f1fa32e/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 682201vm-test-run-nixbot-gitlab> gitlab # [  209.960860] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:28.439Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"98f220b57a44510928ad8593","created_at":"2026-06-14T15:02:28.091Z","trace_propagation_headers":{"sentry-trace":"632aa37fa1ec492a975ce4e583b089d4-84bc648976064805","baggage":"sentry-trace_id=632aa37fa1ec492a975ce4e583b089d4,sentry-release=Unknown"},"correlation_id":"01KV3AFMCNXRMF99GDJ4PE6V54","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:28.095Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-98f220b57a44510928ad8593: start","job_status":"start","queue_duration_s":0.344059,"scheduling_latency_s":0.34408}2202vm-test-run-nixbot-gitlab> gitlab # [  209.961708] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:28.462Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"98f220b57a44510928ad8593","created_at":"2026-06-14T15:02:28.091Z","trace_propagation_headers":{"sentry-trace":"632aa37fa1ec492a975ce4e583b089d4-84bc648976064805","baggage":"sentry-trace_id=632aa37fa1ec492a975ce4e583b089d4,sentry-release=Unknown"},"correlation_id":"01KV3AFMCNXRMF99GDJ4PE6V54","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:28.095Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-98f220b57a44510928ad8593: done: 0.023597 sec","job_status":"done","queue_duration_s":0.344059,"scheduling_latency_s":0.34408,"redis_calls":10,"redis_duration_s":0.0022400000000000002,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000408,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001311,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000521,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014617,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033021,"duration_s":0.023597,"completed_at":"2026-06-14T15:02:28.462Z","load_balancing_strategy":"primary","db_duration_s":0.003798,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2203vm-test-run-nixbot-gitlab> gitlab # [  209.993819] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:28.464Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"c81ec542b1aa24e201ec55e9","created_at":"2026-06-14T15:02:28.137Z","trace_propagation_headers":{"sentry-trace":"632aa37fa1ec492a975ce4e583b089d4-84bc648976064805","baggage":"sentry-trace_id=632aa37fa1ec492a975ce4e583b089d4,sentry-release=Unknown"},"correlation_id":"01KV3AFMCNXRMF99GDJ4PE6V54","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:28.139Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-c81ec542b1aa24e201ec55e9: start","job_status":"start","queue_duration_s":0.325068,"scheduling_latency_s":0.325089}2204vm-test-run-nixbot-gitlab> gitlab # [  210.018955] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:28.787Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"c81ec542b1aa24e201ec55e9","created_at":"2026-06-14T15:02:28.137Z","trace_propagation_headers":{"sentry-trace":"632aa37fa1ec492a975ce4e583b089d4-84bc648976064805","baggage":"sentry-trace_id=632aa37fa1ec492a975ce4e583b089d4,sentry-release=Unknown"},"correlation_id":"01KV3AFMCNXRMF99GDJ4PE6V54","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:28.139Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-c81ec542b1aa24e201ec55e9: done: 0.323777 sec","job_status":"done","queue_duration_s":0.325068,"scheduling_latency_s":0.325089,"gitaly_calls":3,"gitaly_duration_s":0.144374,"redis_calls":44,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.010294,"redis_read_bytes":98,"redis_write_bytes":6013,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001134,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.002727,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":2,"redis_queues_duration_s":0.00044,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1132,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001983,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000485,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.001095,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001296,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":20,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.004,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.14272,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.304220508,"duration_s":0.323777,"completed_at":"2026-06-14T15:02:28.787Z","load_balancing_strategy":"primary","db_duration_s":0.030216,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2205vm-test-run-nixbot-gitlab> gitlab # [  210.103621] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:28.791Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"dbc4c27197c9eaedda4af5b4","created_at":"2026-06-14T15:02:28.569Z","trace_propagation_headers":{"sentry-trace":"632aa37fa1ec492a975ce4e583b089d4-4c4beee9978c402e","baggage":"sentry-trace_id=632aa37fa1ec492a975ce4e583b089d4,sentry-release=Unknown"},"correlation_id":"01KV3AFMCNXRMF99GDJ4PE6V54","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:28.573Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-dbc4c27197c9eaedda4af5b4: start","job_status":"start","queue_duration_s":0.218042,"scheduling_latency_s":0.218065}2206vm-test-run-nixbot-gitlab> gitlab # [  210.124977] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:28.816Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"dbc4c27197c9eaedda4af5b4","created_at":"2026-06-14T15:02:28.569Z","trace_propagation_headers":{"sentry-trace":"632aa37fa1ec492a975ce4e583b089d4-4c4beee9978c402e","baggage":"sentry-trace_id=632aa37fa1ec492a975ce4e583b089d4,sentry-release=Unknown"},"correlation_id":"01KV3AFMCNXRMF99GDJ4PE6V54","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:28.573Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-dbc4c27197c9eaedda4af5b4: done: 0.025493 sec","job_status":"done","queue_duration_s":0.218042,"scheduling_latency_s":0.218065,"redis_calls":7,"redis_duration_s":0.002335,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001878,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000457,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016151,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025493,"completed_at":"2026-06-14T15:02:28.816Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003054,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2207vm-test-run-nixbot-gitlab> gitlab # [  210.240413] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:28.836Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"2b3f7854fb17d55b347eeb99","created_at":"2026-06-14T15:02:08.004Z","trace_propagation_headers":{"sentry-trace":"f44c1b3c364443b48b8813968e1f9412-20b75bdf464041f6","baggage":"sentry-trace_id=f44c1b3c364443b48b8813968e1f9412,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV3AF141KPARZTDRDN91VDP7","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:ec79542482632c5ce9f79bb038a3b15f8e8ac74c733b2f297de7562da91ce3f5","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:28.835Z","retry_count":0,"job_size_bytes":451,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-2b3f7854fb17d55b347eeb99: start","job_status":"start","queue_duration_s":0.00111,"scheduling_latency_s":0.001135}2208vm-test-run-nixbot-gitlab> gitlab # [  210.241527] gitaly[1387]: time="2026-06-14T15:02:28.752Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFMCNXRMF99GDJ4PE6V54 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T15:02:58.751" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:28.751" grpc.time_ms=0.522 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2209vm-test-run-nixbot-gitlab> gitlab # [  210.242503] sidekiq-cluster[2097]: {"severity":"WARN","time":"2026-06-14T15:02:28.861Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"2b3f7854fb17d55b347eeb99","created_at":"2026-06-14T15:02:08.004Z","trace_propagation_headers":{"sentry-trace":"f44c1b3c364443b48b8813968e1f9412-20b75bdf464041f6","baggage":"sentry-trace_id=f44c1b3c364443b48b8813968e1f9412,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV3AF141KPARZTDRDN91VDP7","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:ec79542482632c5ce9f79bb038a3b15f8e8ac74c733b2f297de7562da91ce3f5","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:28.835Z","retry_count":0,"job_size_bytes":451,"pid":2097,"sidekiq_tid":"ma6p","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-2b3f7854fb17d55b347eeb99: fail: 0.024825 sec","job_status":"fail","queue_duration_s":0.00111,"scheduling_latency_s":0.001135,"redis_calls":7,"redis_duration_s":0.001565,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001105,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00046,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016077,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024825,"completed_at":"2026-06-14T15:02:28.861Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.002939,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2210vm-test-run-nixbot-gitlab> gitlab # [  210.841328] gitaly[1387]: time="2026-06-14T15:02:28.777Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFMCNXRMF99GDJ4PE6V54 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T15:02:58.776" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:28.776" grpc.time_ms=0.817 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2211vm-test-run-nixbot-gitlab> gitlab # [  210.860933] gitaly[1387]: time="2026-06-14T15:02:28.788Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFNM5N5KYKXJHE2CX0QRC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T15:02:58.786" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:28.787" grpc.time_ms=0.898 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2212vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/2ee4ce11810ac90f03a8181f7104629d7f1fa32e/statuses, in 1.60 seconds)2213vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '2ee4ce11810ac90f03a8181f7104629d7f1fa32e', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-06-14T15:02:19.830Z', 'started_at': None, 'finished_at': '2026-06-14T15:02:27.684Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '2ee4ce11810ac90f03a8181f7104629d7f1fa32e', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-06-14T15:02:27.834Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2214vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/2ee4ce11810ac90f03a8181f7104629d7f1fa32e/statuses2215vm-test-run-nixbot-gitlab> gitlab # [  212.086877] gitaly[1387]: time="2026-06-14T15:02:31.348Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFR34PX6JM275A1SAZJZ1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T15:03:01.245" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:31.346" grpc.time_ms=1.366 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2216vm-test-run-nixbot-gitlab> gitlab # [  212.121561] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:31 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/2ee4ce11810ac90f03a8181f7104629d7f1fa32e/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 1132217vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/2ee4ce11810ac90f03a8181f7104629d7f1fa32e/statuses, in 0.15 seconds)2218vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '2ee4ce11810ac90f03a8181f7104629d7f1fa32e', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-06-14T15:02:19.830Z', 'started_at': None, 'finished_at': '2026-06-14T15:02:27.684Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '2ee4ce11810ac90f03a8181f7104629d7f1fa32e', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-06-14T15:02:27.834Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2219vm-test-run-nixbot-gitlab> gitlab # [  212.819741] nix-daemon[3369]: accepted connection from pid 3567, user nixbot2220vm-test-run-nixbot-gitlab> gitlab # [  212.888989] gitaly[1387]: time="2026-06-14T15:02:32.151Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFRXC2RMPRPDVQ02PH71C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T15:03:02.148" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:32.149" grpc.time_ms=0.989 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2221vm-test-run-nixbot-gitlab> gitlab # [  212.932141] gitaly[1387]: time="2026-06-14T15:02:32.194Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KV3AFRXC2RMPRPDVQ02PH71C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-14T15:02:42.087" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T15:02:32.188" grpc.time_ms=6.054 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2222vm-test-run-nixbot-gitlab> gitlab # [  212.988385] sidekiq-cluster[2097]: {"severity":"INFO","time":"2026-06-14T15:02:32.249Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"622448bb2d637f2e90045904","created_at":"2026-06-14T15:02:32.241Z","trace_propagation_headers":{"sentry-trace":"5cf7c2dcffa443f8ae324d18938e5e1e-14ed7d97374c4320","baggage":"sentry-trace_id=5cf7c2dcffa443f8ae324d18938e5e1e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV3AFRXC2RMPRPDVQ02PH71C","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-14T15:02:32.244Z","job_size_bytes":3,"pid":2097,"sidekiq_tid":"m7st","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-622448bb2d637f2e90045904: start","job_status":"start","queue_duration_s":0.005407,"scheduling_latency_s":0.005443}2223vm-test-run-nixbot-gitlab> gitlab # [  212.995425] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:32 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/2ee4ce11810ac90f03a8181f7104629d7f1fa32e HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1482224vm-test-run-nixbot-gitlab> gitlab # [  212.997338] nixbot[3073]: {"ts": "2026-06-14T15:02:32.258Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/2ee4ce11810ac90f03a8181f7104629d7f1fa32e \"HTTP/1.1 201 Created\""}2225vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/2ee4ce11810ac90f03a8181f7104629d7f1fa32e/statuses2226vm-test-run-nixbot-gitlab> gitlab # [  213.178200] nix-daemon[3369]: accepted connection from pid 3582, user nixbot2227vm-test-run-nixbot-gitlab> gitlab # [  213.199373] gitaly[1387]: time="2026-06-14T15:02:32.460Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV3AFRXC2RMPRPDVQ02PH71C grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T15:02:32.460" grpc.time_ms=0.457 pid=1387 span.kind=server system=grpc2228vm-test-run-nixbot-gitlab> gitlab # [  213.226883] gitaly[1387]: time="2026-06-14T15:02:32.488Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFS75Q7Y83NNHJAHVHPC5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-14T15:03:02.486" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-14T15:02:32.487" grpc.time_ms=1.073 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2229vm-test-run-nixbot-gitlab> gitlab # [  213.240377] gitaly[1387]: time="2026-06-14T15:02:32.502Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV3AFRXC2RMPRPDVQ02PH71C grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T15:02:32.502" grpc.time_ms=0.369 pid=1387 span.kind=server system=grpc2230vm-test-run-nixbot-gitlab> gitlab # [  213.288547] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/06/14:15:02:32 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/2ee4ce11810ac90f03a8181f7104629d7f1fa32e/statuses HTTP/1.1" 200 1328 "" "curl/8.20.0" 1262231vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/2ee4ce11810ac90f03a8181f7104629d7f1fa32e/statuses, in 0.17 seconds)2232vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '2ee4ce11810ac90f03a8181f7104629d7f1fa32e', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-06-14T15:02:19.830Z', 'started_at': None, 'finished_at': '2026-06-14T15:02:27.684Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '2ee4ce11810ac90f03a8181f7104629d7f1fa32e', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-06-14T15:02:27.834Z', 'started_at': None, 'finished_at': '2026-06-14T15:02:32.233Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2233vm-test-run-nixbot-gitlab> (finished: subtest: eval and build post commit statuses, in 9.68 seconds)2234vm-test-run-nixbot-gitlab> (finished: run the VM test script, in 255.59 seconds)2235vm-test-run-nixbot-gitlab> gitlab # [  213.301869] gitaly[1387]: time="2026-06-14T15:02:32.564Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV3AFRXC2RMPRPDVQ02PH71C grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-14T15:02:32.563" grpc.time_ms=0.328 pid=1387 span.kind=server system=grpc2236vm-test-run-nixbot-gitlab> gitlab # [  213.307791] gitaly[1387]: time="2026-06-14T15:02:32.570Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFRXC2RMPRPDVQ02PH71C forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-14T15:03:02.431" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-14T15:02:32.432" operations="[repack_refs]" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2237vm-test-run-nixbot-gitlab> gitlab # [  213.312777] gitaly[1387]: time="2026-06-14T15:02:32.573Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFRXC2RMPRPDVQ02PH71C grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-14T15:03:02.431" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-14T15:02:32.432" pid=1387 remote_ip="2001:db8:1::1" repository_info="{false {6 703 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-06-14 15:02:09.581633818 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root2238vm-test-run-nixbot-gitlab> gitlab # [  213.318625] gitaly[1387]: time="2026-06-14T15:02:32.580Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFRXC2RMPRPDVQ02PH71C diskcache=524dcde4-9c44-43ac-a195-fc0d12503465 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-14T15:03:02.431" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-14T15:02:32.432" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2239vm-test-run-nixbot-gitlab> gitlab # [  213.321698] gitaly[1387]: time="2026-06-14T15:02:32.583Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=104 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7518 command.oublock=0 command.real_time_ms=140 command.spawn_token_fork_ms=4 command.system_time_ms=77 command.user_time_ms=27 component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFRXC2RMPRPDVQ02PH71C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-14T15:03:02.431" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-14T15:02:32.432" grpc.time_ms=151.45 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2240vm-test-run-nixbot-gitlab> gitlab # [  213.330178] gitaly[1387]: time="2026-06-14T15:02:32.592Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFRXC2RMPRPDVQ02PH71C grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-14T15:03:02.431" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-14T15:02:32.432" pid=1387 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root2241vm-test-run-nixbot-gitlab> gitlab # [  213.407292] gitaly[1387]: time="2026-06-14T15:02:32.592Z" level=info msg="optimized repository" clean_stale_data_time=18.160688ms clean_work_trees_time="84.368µs" commit_graph_time="7.543µs" correlation_id=01KV3AFRXC2RMPRPDVQ02PH71C gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="8.102µs" pid=1387 prune_time="7.543µs" relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="8.102µs" storage_name=default system=housekeeping total_time=18.473856ms2242vm-test-run-nixbot-gitlab> gitlab # [  213.408848] gitaly[1387]: time="2026-06-14T15:02:32.592Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01KV3AFRXC2RMPRPDVQ02PH71C grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-14T15:03:02.431" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-14T15:02:32.432" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2243vm-test-run-nixbot-gitlab> test script finished in 255.75s2244vm-test-run-nixbot-gitlab> cleanup2245vm-test-run-nixbot-gitlab> kill QemuMachine (pid 13)2246vm-test-run-nixbot-gitlab> gitlab # qemu-system-x86_64: terminating on signal 15 from pid 6 (/nix/store/60m4rxhg2fldqaak400c0lry96ijrzqn-python3-3.13.13/bin/python3.13)2247vm-test-run-nixbot-gitlab> (finished: cleanup, in 1.33 seconds)22482249post-build step Upload coverage to codecov: ok2250Skipping codecov: project=Mic92/nixbot attr=x86_64-linux.nixbot-gitlab