nixbot

1these 13 derivations will be built:2  /nix/store/kly7bijyz1s9ka21q589wl8jqffdbnz8-python3.13-nixbot.drv3  /nix/store/jq2f7xwcqcqs5bk16jhiq38cnk6dm0bq-unit-nixbot.service.drv4  /nix/store/aacxx4hj97qyjfdh53mgixv2p2yl9fpv-system-units.drv5  /nix/store/514zinvy5j4x83j1g0faalnkf3lbz54q-etc.drv6  /nix/store/gbrm067svyj5wf0aff2cmq0d4lzv87qy-activate.drv7  /nix/store/6nl9468i6qlzkqmmdkm1ap14wgakrng2-nixos-system-gitlab-test.drv8  /nix/store/c9h5b7cq0d6wrj2vwbhqz182bckv9vd7-closure-info.drv9  /nix/store/pb1v6330xvwxba71qknshm7nmsrnav79-closure-info.drv10  /nix/store/55bya7i95awaam6cw83pcb60aymbn342-run-nixos-vm.drv11  /nix/store/122yhz46mycihcv6snl63bhya8rgindk-nixos-vm.drv12  /nix/store/65r4fvflb3wfmrxk6b88c5sf8s47m60b-driverConfiguration.json.drv13  /nix/store/2w6pb3j5dgslfb5vr6y1q631hsvw3k6i-nixos-test-driver-nixbot-gitlab.drv14  /nix/store/a8c4sk4xqk55nwn1mvcb3wfdapr44h77-vm-test-run-nixbot-gitlab.drv15building '/nix/store/kly7bijyz1s9ka21q589wl8jqffdbnz8-python3.13-nixbot.drv' on 'ssh-ng://nix@jamie'16building '/nix/store/kly7bijyz1s9ka21q589wl8jqffdbnz8-python3.13-nixbot.drv'17python3.13-nixbot> Sourcing python-remove-tests-dir-hook18python3.13-nixbot> Sourcing python-catch-conflicts-hook.sh19python3.13-nixbot> Sourcing python-remove-bin-bytecode-hook.sh20python3.13-nixbot> Sourcing pypa-build-hook21python3.13-nixbot> Using pypaBuildPhase22python3.13-nixbot> Sourcing python-runtime-deps-check-hook23python3.13-nixbot> Using pythonRuntimeDepsCheckHook24python3.13-nixbot> Sourcing pypa-install-hook25python3.13-nixbot> Using pypaInstallPhase26python3.13-nixbot> Sourcing python-imports-check-hook.sh27python3.13-nixbot> Using pythonImportsCheckPhase28python3.13-nixbot> Sourcing python-namespaces-hook29python3.13-nixbot> Sourcing python-catch-conflicts-hook.sh30python3.13-nixbot> Running phase: unpackPhase31python3.13-nixbot> unpacking source archive /nix/store/njg31nkihak4vblnn3v9dhw2j8wiwzyp-nixbot32python3.13-nixbot> source root is nixbot33python3.13-nixbot> setting SOURCE_DATE_EPOCH to timestamp 315619200 of file "nixbot/pyproject.toml"34python3.13-nixbot> Running phase: patchPhase35python3.13-nixbot> Running phase: updateAutotoolsGnuConfigScriptsPhase36python3.13-nixbot> Running phase: configurePhase37python3.13-nixbot> no configure script, doing nothing38python3.13-nixbot> Running phase: buildPhase39python3.13-nixbot> Executing pypaBuildPhase40python3.13-nixbot> Creating a wheel...41python3.13-nixbot> pypa build flags: --no-isolation --outdir dist/ --wheel42python3.13-nixbot> * Getting build dependencies for wheel...43python3.13-nixbot> * Building wheel...44python3.13-nixbot> Successfully built nixbot-0.0.1-py3-none-any.whl45python3.13-nixbot> Finished creating a wheel...46python3.13-nixbot> Finished executing pypaBuildPhase47python3.13-nixbot> Running phase: pythonRuntimeDepsCheckHook48python3.13-nixbot> Executing pythonRuntimeDepsCheck49python3.13-nixbot> Checking runtime dependencies for nixbot-0.0.1-py3-none-any.whl50python3.13-nixbot> Finished executing pythonRuntimeDepsCheck51python3.13-nixbot> Running phase: installPhase52python3.13-nixbot> Executing pypaInstallPhase53python3.13-nixbot> Successfully installed nixbot-0.0.1-py3-none-any.whl54python3.13-nixbot> Finished executing pypaInstallPhase55python3.13-nixbot> Running phase: pythonOutputDistPhase56python3.13-nixbot> Executing pythonOutputDistPhase57python3.13-nixbot> Finished executing pythonOutputDistPhase58python3.13-nixbot> Running phase: fixupPhase59python3.13-nixbot> shrinking RPATHs of ELF executables and libraries in /nix/store/ihhc5jvzgxf9963i8cxl662995sh2qkh-python3.13-nixbot60python3.13-nixbot> checking for references to /build/ in /nix/store/ihhc5jvzgxf9963i8cxl662995sh2qkh-python3.13-nixbot...61python3.13-nixbot> patching script interpreter paths in /nix/store/ihhc5jvzgxf9963i8cxl662995sh2qkh-python3.13-nixbot62python3.13-nixbot> stripping (with command strip and flags -S -p) in  /nix/store/ihhc5jvzgxf9963i8cxl662995sh2qkh-python3.13-nixbot/lib /nix/store/ihhc5jvzgxf9963i8cxl662995sh2qkh-python3.13-nixbot/bin63python3.13-nixbot> shrinking RPATHs of ELF executables and libraries in /nix/store/60ljaxpwm5wp6ncm8zbiiylik0xmdc97-python3.13-nixbot-dist64python3.13-nixbot> checking for references to /build/ in /nix/store/60ljaxpwm5wp6ncm8zbiiylik0xmdc97-python3.13-nixbot-dist...65python3.13-nixbot> patching script interpreter paths in /nix/store/60ljaxpwm5wp6ncm8zbiiylik0xmdc97-python3.13-nixbot-dist66python3.13-nixbot> Rewriting #!/nix/store/60m4rxhg2fldqaak400c0lry96ijrzqn-python3-3.13.13/bin/python3.13 to #!/nix/store/60m4rxhg2fldqaak400c0lry96ijrzqn-python3-3.13.1367python3.13-nixbot> wrapping `/nix/store/ihhc5jvzgxf9963i8cxl662995sh2qkh-python3.13-nixbot/bin/nixbot'...68python3.13-nixbot> Executing pythonRemoveTestsDir69python3.13-nixbot> Finished executing pythonRemoveTestsDir70python3.13-nixbot> Running phase: pythonCatchConflictsPhase71python3.13-nixbot> Running phase: pythonRemoveBinBytecodePhase72python3.13-nixbot> Running phase: pythonImportsCheckPhase73python3.13-nixbot> Executing pythonImportsCheckPhase74building '/nix/store/jq2f7xwcqcqs5bk16jhiq38cnk6dm0bq-unit-nixbot.service.drv'75building '/nix/store/aacxx4hj97qyjfdh53mgixv2p2yl9fpv-system-units.drv'76building '/nix/store/514zinvy5j4x83j1g0faalnkf3lbz54q-etc.drv'77building '/nix/store/gbrm067svyj5wf0aff2cmq0d4lzv87qy-activate.drv'78building '/nix/store/6nl9468i6qlzkqmmdkm1ap14wgakrng2-nixos-system-gitlab-test.drv'79nixos-system-gitlab-test> structuredAttrs is enabled80building '/nix/store/c9h5b7cq0d6wrj2vwbhqz182bckv9vd7-closure-info.drv'81closure-info> structuredAttrs is enabled82building '/nix/store/pb1v6330xvwxba71qknshm7nmsrnav79-closure-info.drv'83closure-info> structuredAttrs is enabled84building '/nix/store/55bya7i95awaam6cw83pcb60aymbn342-run-nixos-vm.drv'85building '/nix/store/122yhz46mycihcv6snl63bhya8rgindk-nixos-vm.drv'86building '/nix/store/65r4fvflb3wfmrxk6b88c5sf8s47m60b-driverConfiguration.json.drv'87driverConfiguration.json> structuredAttrs is enabled88building '/nix/store/2w6pb3j5dgslfb5vr6y1q631hsvw3k6i-nixos-test-driver-nixbot-gitlab.drv'89nixos-test-driver-nixbot-gitlab> Running type check (enable/disable: config.skipTypeCheck)90nixos-test-driver-nixbot-gitlab> See https://nixos.org/manual/nixos/stable/#test-opt-skipTypeCheck91nixos-test-driver-nixbot-gitlab> All checks passed!92nixos-test-driver-nixbot-gitlab> Linting test script (enable/disable: config.skipLint)93nixos-test-driver-nixbot-gitlab> See https://nixos.org/manual/nixos/stable/#test-opt-skipLint94nixos-test-driver-nixbot-gitlab> All checks passed!95building '/nix/store/a8c4sk4xqk55nwn1mvcb3wfdapr44h77-vm-test-run-nixbot-gitlab.drv' on 'ssh-ng://nix@jamie'96building '/nix/store/a8c4sk4xqk55nwn1mvcb3wfdapr44h77-vm-test-run-nixbot-gitlab.drv'97vm-test-run-nixbot-gitlab> Machine state will be reset. To keep it, pass --keep-machine-state98vm-test-run-nixbot-gitlab> start all VLans99vm-test-run-nixbot-gitlab> (finished: start all VLans, in 0.00 seconds)100vm-test-run-nixbot-gitlab> Test will time out and terminate in 3600 seconds101vm-test-run-nixbot-gitlab> run the VM test script102vm-test-run-nixbot-gitlab> additionally exposed symbols:103vm-test-run-nixbot-gitlab>     gitlab,104vm-test-run-nixbot-gitlab>     vlan1,105vm-test-run-nixbot-gitlab>     start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh106vm-test-run-nixbot-gitlab> start all VMs107vm-test-run-nixbot-gitlab> gitlab: starting vm108vm-test-run-nixbot-gitlab> gitlab # Disk image does not exist, creating the virtualisation disk image...109vm-test-run-nixbot-gitlab> gitlab: QEMU running (pid 13)110vm-test-run-nixbot-gitlab> gitlab # Formatting '/build/vm-state-gitlab/tmp.R4tdy8wH2M', fmt=raw size=1073741824111vm-test-run-nixbot-gitlab> gitlab # mke2fs 1.47.3 (8-Jul-2025)112vm-test-run-nixbot-gitlab> gitlab # Discarding device blocks:      0/262144             done113vm-test-run-nixbot-gitlab> gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes114vm-test-run-nixbot-gitlab> gitlab # Filesystem UUID: 007f89d7-2247-44f5-8ffd-793483f8687d115vm-test-run-nixbot-gitlab> gitlab # Superblock backups stored on blocks:116vm-test-run-nixbot-gitlab> gitlab # 	32768, 98304, 163840, 229376117vm-test-run-nixbot-gitlab> (finished: start all VMs, in 34.13 seconds)118vm-test-run-nixbot-gitlab> gitlab # 119vm-test-run-nixbot-gitlab> gitlab: waiting for unit setup-gitlab.service120vm-test-run-nixbot-gitlab> gitlab # Allocating group tables: 0/8   done121vm-test-run-nixbot-gitlab> gitlab # Writing inode tables: 0/8   done122vm-test-run-nixbot-gitlab> gitlab: waiting for the VM to finish booting123vm-test-run-nixbot-gitlab> gitlab # Creating journal (8192 blocks): done124vm-test-run-nixbot-gitlab> gitlab # Writing superblocks and filesystem accounting information: 0/8   done125vm-test-run-nixbot-gitlab> gitlab # 126vm-test-run-nixbot-gitlab> gitlab # Virtualisation disk image created.127vm-test-run-nixbot-gitlab> gitlab # Creating Nix store image...128vm-test-run-nixbot-gitlab> gitlab # Created Nix store image.129vm-test-run-nixbot-gitlab> gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)130vm-test-run-nixbot-gitlab> gitlab # 131vm-test-run-nixbot-gitlab> gitlab # 132vm-test-run-nixbot-gitlab> gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA00133vm-test-run-nixbot-gitlab> gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)...134vm-test-run-nixbot-gitlab> gitlab # 135vm-test-run-nixbot-gitlab> gitlab # 136vm-test-run-nixbot-gitlab> gitlab # 137vm-test-run-nixbot-gitlab> gitlab # 138vm-test-run-nixbot-gitlab> gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB00139vm-test-run-nixbot-gitlab> gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)...140vm-test-run-nixbot-gitlab> gitlab # 141vm-test-run-nixbot-gitlab> gitlab # 142vm-test-run-nixbot-gitlab> gitlab # Booting from ROM...143vm-test-run-nixbot-gitlab> gitlab # Probing EDD (edd=off to disable)... ok[    0.000000] Linux version 6.18.34 (nixbld@localhost) (gcc (GCC) 15.2.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Jun  1 15:51:08 UTC 2026144vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/yy7030swsiwlxjxgzf612g3r62cyp71p-nixos-system-gitlab-test/init regInfo=/nix/store/r2g8i79k3rnn4and6yyivl1m5g256dc5-closure-info/registration console=ttyS0,115200n8 console=tty0145vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-provided physical RAM map:146vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable147vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved148vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved149vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable150vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved151vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved152vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved153vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable154vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved155vm-test-run-nixbot-gitlab> gitlab # [    0.000000] NX (Execute Disable) protection: active156vm-test-run-nixbot-gitlab> gitlab # [    0.000000] APIC: Static calls initialized157vm-test-run-nixbot-gitlab> gitlab # [    0.000000] SMBIOS 2.8 present.158vm-test-run-nixbot-gitlab> gitlab # [    0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014159vm-test-run-nixbot-gitlab> gitlab # [    0.000000] DMI: Memory slots populated: 1/1160vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Hypervisor detected: KVM161vm-test-run-nixbot-gitlab> gitlab # [    0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000162vm-test-run-nixbot-gitlab> gitlab # [    0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00163vm-test-run-nixbot-gitlab> gitlab # [    0.000000] kvm-clock: using sched offset of 504391391 cycles164vm-test-run-nixbot-gitlab> gitlab # [    0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns165vm-test-run-nixbot-gitlab> gitlab # [    0.000005] tsc: Detected 2400.010 MHz processor166vm-test-run-nixbot-gitlab> gitlab # [    0.000810] last_pfn = 0x1c0000 max_arch_pfn = 0x10000000000167vm-test-run-nixbot-gitlab> gitlab # [    0.000847] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs168vm-test-run-nixbot-gitlab> gitlab # [    0.000849] x86/PAT: Configuration [0-7]: WB  WC  UC- UC  WB  WP  UC- WT169vm-test-run-nixbot-gitlab> gitlab # [    0.000900] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000170vm-test-run-nixbot-gitlab> gitlab # [    0.002747] found SMP MP-table at [mem 0x000f5470-0x000f547f]171vm-test-run-nixbot-gitlab> gitlab # [    0.002759] Using GB pages for direct mapping172vm-test-run-nixbot-gitlab> gitlab # [    0.002942] RAMDISK: [mem 0xbe4e0000-0xbffcffff]173vm-test-run-nixbot-gitlab> gitlab # [    0.002950] ACPI: Early table checksum verification disabled174vm-test-run-nixbot-gitlab> gitlab # [    0.002953] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )175vm-test-run-nixbot-gitlab> gitlab # [    0.002957] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS  BXPC     00000001 BXPC 00000001)176vm-test-run-nixbot-gitlab> gitlab # [    0.002961] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS  BXPC     00000001 BXPC 00000001)177vm-test-run-nixbot-gitlab> gitlab # [    0.002967] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS  BXPC     00000001 BXPC 00000001)178vm-test-run-nixbot-gitlab> gitlab # [    0.002969] ACPI: FACS 0x00000000BFFE0000 000040179vm-test-run-nixbot-gitlab> gitlab # [    0.002970] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS  BXPC     00000001 BXPC 00000001)180vm-test-run-nixbot-gitlab> gitlab # [    0.002972] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS  BXPC     00000001 BXPC 00000001)181vm-test-run-nixbot-gitlab> gitlab # [    0.002974] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS  BXPC     00000001 BXPC 00000001)182vm-test-run-nixbot-gitlab> gitlab # [    0.002975] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2]183vm-test-run-nixbot-gitlab> gitlab # [    0.002976] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e]184vm-test-run-nixbot-gitlab> gitlab # [    0.002977] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f]185vm-test-run-nixbot-gitlab> gitlab # [    0.002977] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482]186vm-test-run-nixbot-gitlab> gitlab # [    0.002978] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba]187vm-test-run-nixbot-gitlab> gitlab # [    0.002978] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2]188vm-test-run-nixbot-gitlab> gitlab # [    0.003455] No NUMA configuration found189vm-test-run-nixbot-gitlab> gitlab # [    0.003456] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff]190vm-test-run-nixbot-gitlab> gitlab # [    0.003459] NODE_DATA(0) allocated [mem 0x1bfff8780-0x1bfffdcff]191vm-test-run-nixbot-gitlab> gitlab # [    0.006119] Zone ranges:192vm-test-run-nixbot-gitlab> gitlab # [    0.006119]   DMA      [mem 0x0000000000001000-0x0000000000ffffff]193vm-test-run-nixbot-gitlab> gitlab # [    0.006121]   DMA32    [mem 0x0000000001000000-0x00000000ffffffff]194vm-test-run-nixbot-gitlab> gitlab # [    0.006122]   Normal   [mem 0x0000000100000000-0x00000001bfffffff]195vm-test-run-nixbot-gitlab> gitlab # [    0.006123]   Device   empty196vm-test-run-nixbot-gitlab> gitlab # [    0.006124] Movable zone start for each node197vm-test-run-nixbot-gitlab> gitlab # [    0.006124] Early memory node ranges198vm-test-run-nixbot-gitlab> gitlab # [    0.006125]   node   0: [mem 0x0000000000001000-0x000000000009efff]199vm-test-run-nixbot-gitlab> gitlab # [    0.006126]   node   0: [mem 0x0000000000100000-0x00000000bffd7fff]200vm-test-run-nixbot-gitlab> gitlab # [    0.006127]   node   0: [mem 0x0000000100000000-0x00000001bfffffff]201vm-test-run-nixbot-gitlab> gitlab # [    0.006128] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff]202vm-test-run-nixbot-gitlab> gitlab # [    0.006147] On node 0, zone DMA: 1 pages in unavailable ranges203vm-test-run-nixbot-gitlab> gitlab # [    0.006421] On node 0, zone DMA: 97 pages in unavailable ranges204vm-test-run-nixbot-gitlab> gitlab # [    0.118184] On node 0, zone Normal: 40 pages in unavailable ranges205vm-test-run-nixbot-gitlab> gitlab # [    0.119208] ACPI: PM-Timer IO Port: 0x608206vm-test-run-nixbot-gitlab> gitlab # [    0.119226] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])207vm-test-run-nixbot-gitlab> gitlab # [    0.119264] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23208vm-test-run-nixbot-gitlab> gitlab # [    0.119267] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)209vm-test-run-nixbot-gitlab> gitlab # [    0.119269] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)210vm-test-run-nixbot-gitlab> gitlab # [    0.119270] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)211vm-test-run-nixbot-gitlab> gitlab # [    0.119271] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)212vm-test-run-nixbot-gitlab> gitlab # [    0.119272] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)213vm-test-run-nixbot-gitlab> gitlab # [    0.119274] ACPI: Using ACPI (MADT) for SMP configuration information214vm-test-run-nixbot-gitlab> gitlab # [    0.119275] ACPI: HPET id: 0x8086a201 base: 0xfed00000215vm-test-run-nixbot-gitlab> gitlab # [    0.119282] TSC deadline timer available216vm-test-run-nixbot-gitlab> gitlab # [    0.119286] CPU topo: Max. logical packages:   1217vm-test-run-nixbot-gitlab> gitlab # [    0.119287] CPU topo: Max. logical dies:       1218vm-test-run-nixbot-gitlab> gitlab # [    0.119287] CPU topo: Max. dies per package:   1219vm-test-run-nixbot-gitlab> gitlab # [    0.119290] CPU topo: Max. threads per core:   1220vm-test-run-nixbot-gitlab> gitlab # [    0.119291] CPU topo: Num. cores per package:     4221vm-test-run-nixbot-gitlab> gitlab # [    0.119292] CPU topo: Num. threads per package:   4222vm-test-run-nixbot-gitlab> gitlab # [    0.119292] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs223vm-test-run-nixbot-gitlab> gitlab # [    0.119316] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()224vm-test-run-nixbot-gitlab> gitlab # [    0.119335] kvm-guest: KVM setup pv remote TLB flush225vm-test-run-nixbot-gitlab> gitlab # [    0.119338] kvm-guest: setup PV sched yield226vm-test-run-nixbot-gitlab> gitlab # [    0.119361] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]227vm-test-run-nixbot-gitlab> gitlab # [    0.119362] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]228vm-test-run-nixbot-gitlab> gitlab # [    0.119363] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff]229vm-test-run-nixbot-gitlab> gitlab # [    0.119365] [mem 0xc0000000-0xfeffbfff] available for PCI devices230vm-test-run-nixbot-gitlab> gitlab # [    0.119366] Booting paravirtualized kernel on KVM231vm-test-run-nixbot-gitlab> gitlab # [    0.119370] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns232vm-test-run-nixbot-gitlab> gitlab # [    0.123819] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1233vm-test-run-nixbot-gitlab> gitlab # [    0.126163] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288234vm-test-run-nixbot-gitlab> gitlab # [    0.126213] kvm-guest: PV spinlocks enabled235vm-test-run-nixbot-gitlab> gitlab # [    0.126215] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear)236vm-test-run-nixbot-gitlab> gitlab # [    0.126218] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/yy7030swsiwlxjxgzf612g3r62cyp71p-nixos-system-gitlab-test/init regInfo=/nix/store/r2g8i79k3rnn4and6yyivl1m5g256dc5-closure-info/registration console=ttyS0,115200n8 console=tty0237vm-test-run-nixbot-gitlab> gitlab # [    0.126312] Unknown kernel command line parameters "regInfo=/nix/store/r2g8i79k3rnn4and6yyivl1m5g256dc5-closure-info/registration", will be passed to user space.238vm-test-run-nixbot-gitlab> gitlab # [    0.126325] random: crng init done239vm-test-run-nixbot-gitlab> gitlab # [    0.126325] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes240vm-test-run-nixbot-gitlab> gitlab # [    0.135516] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)241vm-test-run-nixbot-gitlab> gitlab # [    0.140226] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)242vm-test-run-nixbot-gitlab> gitlab # [    0.140302] software IO TLB: area num 4.243vm-test-run-nixbot-gitlab> gitlab # [    0.213552] Fallback order for Node 0: 0244vm-test-run-nixbot-gitlab> gitlab # [    0.213559] Built 1 zonelists, mobility grouping on.  Total pages: 1572726245vm-test-run-nixbot-gitlab> gitlab # [    0.213560] Policy zone: Normal246vm-test-run-nixbot-gitlab> gitlab # [    0.216080] mem auto-init: stack:all(zero), heap alloc:on, heap free:off247vm-test-run-nixbot-gitlab> gitlab # [    0.225629] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1248vm-test-run-nixbot-gitlab> gitlab # [    0.239236] allocated 12582912 bytes of page_ext249vm-test-run-nixbot-gitlab> gitlab # [    0.249229] ftrace: allocating 48565 entries in 192 pages250vm-test-run-nixbot-gitlab> gitlab # [    0.249231] ftrace: allocated 192 pages with 2 groups251vm-test-run-nixbot-gitlab> gitlab # [    0.250068] Dynamic Preempt: lazy252vm-test-run-nixbot-gitlab> gitlab # [    0.250235] rcu: Preemptible hierarchical RCU implementation.253vm-test-run-nixbot-gitlab> gitlab # [    0.250236] rcu: 	RCU event tracing is enabled.254vm-test-run-nixbot-gitlab> gitlab # [    0.250237] rcu: 	RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.255vm-test-run-nixbot-gitlab> gitlab # [    0.250238] 	Trampoline variant of Tasks RCU enabled.256vm-test-run-nixbot-gitlab> gitlab # [    0.250239] 	Rude variant of Tasks RCU enabled.257vm-test-run-nixbot-gitlab> gitlab # [    0.250239] 	Tracing variant of Tasks RCU enabled.258vm-test-run-nixbot-gitlab> gitlab # [    0.250240] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.259vm-test-run-nixbot-gitlab> gitlab # [    0.250240] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4260vm-test-run-nixbot-gitlab> gitlab # [    0.250261] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.261vm-test-run-nixbot-gitlab> gitlab # [    0.250262] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.262vm-test-run-nixbot-gitlab> gitlab # [    0.250264] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.263vm-test-run-nixbot-gitlab> gitlab # [    0.254622] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16264vm-test-run-nixbot-gitlab> gitlab # [    0.254944] rcu: srcu_init: Setting srcu_struct sizes based on contention.265vm-test-run-nixbot-gitlab> gitlab # [    0.255062] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)266vm-test-run-nixbot-gitlab> gitlab # [    0.262386] Console: colour VGA+ 80x25267vm-test-run-nixbot-gitlab> gitlab # [    0.262390] printk: legacy console [tty0] enabled268vm-test-run-nixbot-gitlab> gitlab # [    0.306249] printk: legacy console [ttyS0] enabled269vm-test-run-nixbot-gitlab> gitlab # [    0.505375] ACPI: Core revision 20250807270vm-test-run-nixbot-gitlab> gitlab # [    0.506909] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns271vm-test-run-nixbot-gitlab> gitlab # [    0.509766] APIC: Switch to symmetric I/O mode setup272vm-test-run-nixbot-gitlab> gitlab # [    0.511521] x2apic enabled273vm-test-run-nixbot-gitlab> gitlab # [    0.512747] APIC: Switched APIC routing to: physical x2apic274vm-test-run-nixbot-gitlab> gitlab # [    0.514499] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask()275vm-test-run-nixbot-gitlab> gitlab # [    0.516727] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself()276vm-test-run-nixbot-gitlab> gitlab # [    0.519484] kvm-guest: setup PV IPIs277vm-test-run-nixbot-gitlab> gitlab # [    0.523212] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1278vm-test-run-nixbot-gitlab> gitlab # [    0.525109] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229840d1729, max_idle_ns: 440795276167 ns279vm-test-run-nixbot-gitlab> gitlab # [    0.528217] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400010)280vm-test-run-nixbot-gitlab> gitlab # [    0.530273] x86/cpu: User Mode Instruction Prevention (UMIP) activated281vm-test-run-nixbot-gitlab> gitlab # [    0.532365] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127282vm-test-run-nixbot-gitlab> gitlab # [    0.533217] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0283vm-test-run-nixbot-gitlab> gitlab # [    0.535221] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto284vm-test-run-nixbot-gitlab> gitlab # [    0.537217] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl285vm-test-run-nixbot-gitlab> gitlab # [    0.538217] Transient Scheduler Attacks: Mitigation: Clear CPU buffers286vm-test-run-nixbot-gitlab> gitlab # [    0.540216] Spectre V2 : Mitigation: Enhanced / Automatic IBRS287vm-test-run-nixbot-gitlab> gitlab # [    0.542216] Speculative Return Stack Overflow: Mitigation: Safe RET288vm-test-run-nixbot-gitlab> gitlab # [    0.543216] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization289vm-test-run-nixbot-gitlab> gitlab # [    0.545216] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier290vm-test-run-nixbot-gitlab> gitlab # [    0.546217] active return thunk: srso_alias_return_thunk291vm-test-run-nixbot-gitlab> gitlab # [    0.547245] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'292vm-test-run-nixbot-gitlab> gitlab # [    0.548216] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'293vm-test-run-nixbot-gitlab> gitlab # [    0.549217] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'294vm-test-run-nixbot-gitlab> gitlab # [    0.550216] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'295vm-test-run-nixbot-gitlab> gitlab # [    0.551216] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'296vm-test-run-nixbot-gitlab> gitlab # [    0.552216] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'297vm-test-run-nixbot-gitlab> gitlab # [    0.553216] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'298vm-test-run-nixbot-gitlab> gitlab # [    0.554217] x86/fpu: Supporting XSAVE feature 0x800: 'Control-flow User registers'299vm-test-run-nixbot-gitlab> gitlab # [    0.555216] x86/fpu: Supporting XSAVE feature 0x1000: 'Control-flow Kernel registers (KVM only)'300vm-test-run-nixbot-gitlab> gitlab # [    0.557217] x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256301vm-test-run-nixbot-gitlab> gitlab # [    0.558216] x86/fpu: xstate_offset[5]:  832, xstate_sizes[5]:   64302vm-test-run-nixbot-gitlab> gitlab # [    0.559216] x86/fpu: xstate_offset[6]:  896, xstate_sizes[6]:  512303vm-test-run-nixbot-gitlab> gitlab # [    0.561216] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024304vm-test-run-nixbot-gitlab> gitlab # [    0.562216] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]:    8305vm-test-run-nixbot-gitlab> gitlab # [    0.563216] x86/fpu: xstate_offset[11]: 2440, xstate_sizes[11]:   16306vm-test-run-nixbot-gitlab> gitlab # [    0.564216] x86/fpu: xstate_offset[12]: 2456, xstate_sizes[12]:   24307vm-test-run-nixbot-gitlab> gitlab # [    0.565216] x86/fpu: Enabled xstate features 0x1ae7, context size is 2480 bytes, using 'compacted' format.308vm-test-run-nixbot-gitlab> gitlab # [    0.595235] Freeing SMP alternatives memory: 44K309vm-test-run-nixbot-gitlab> gitlab # [    0.596219] pid_max: default: 32768 minimum: 301310vm-test-run-nixbot-gitlab> gitlab # [    0.598240] LSM: initializing lsm=capability,landlock,yama,bpf,ima311vm-test-run-nixbot-gitlab> gitlab # [    0.599330] landlock: Up and running.312vm-test-run-nixbot-gitlab> gitlab # [    0.601216] Yama: becoming mindful.313vm-test-run-nixbot-gitlab> gitlab # [    0.602427] LSM support for eBPF active314vm-test-run-nixbot-gitlab> gitlab # [    0.604225] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)315vm-test-run-nixbot-gitlab> gitlab # [    0.605365] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)316vm-test-run-nixbot-gitlab> gitlab # [    0.608119] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)317vm-test-run-nixbot-gitlab> gitlab # [    0.608811] Performance Events: Fam17h+ core perfctr, AMD PMU driver.318vm-test-run-nixbot-gitlab> gitlab # [    0.609221] ... version:                   2319vm-test-run-nixbot-gitlab> gitlab # [    0.610219] ... bit width:                 48320vm-test-run-nixbot-gitlab> gitlab # [    0.611231] ... generic counters:          6321vm-test-run-nixbot-gitlab> gitlab # [    0.612218] ... generic bitmap:            000000000000003f322vm-test-run-nixbot-gitlab> gitlab # [    0.613218] ... fixed-purpose counters:    0323vm-test-run-nixbot-gitlab> gitlab # [    0.614218] ... fixed-purpose bitmap:      0000000000000000324vm-test-run-nixbot-gitlab> gitlab # [    0.615218] ... value mask:                0000ffffffffffff325vm-test-run-nixbot-gitlab> gitlab # [    0.616218] ... max period:                00007fffffffffff326vm-test-run-nixbot-gitlab> gitlab # [    0.617218] ... global_ctrl mask:          000000000000003f327vm-test-run-nixbot-gitlab> gitlab # [    0.618344] signal: max sigframe size: 3376328vm-test-run-nixbot-gitlab> gitlab # [    0.619324] rcu: Hierarchical SRCU implementation.329vm-test-run-nixbot-gitlab> gitlab # [    0.620223] rcu: 	Max phase no-delay instances is 400.330vm-test-run-nixbot-gitlab> gitlab # [    0.621417] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level331vm-test-run-nixbot-gitlab> gitlab # [    0.626757] smp: Bringing up secondary CPUs ...332vm-test-run-nixbot-gitlab> gitlab # [    0.627689] smpboot: x86: Booting SMP configuration:333vm-test-run-nixbot-gitlab> gitlab # [    0.628221] .... node  #0, CPUs:      #1 #2 #3334vm-test-run-nixbot-gitlab> gitlab # [    0.632362] smp: Brought up 1 node, 4 CPUs335vm-test-run-nixbot-gitlab> gitlab # [    0.634218] smpboot: Total of 4 processors activated (19200.08 BogoMIPS)336vm-test-run-nixbot-gitlab> gitlab # [    0.635792] Memory: 6013360K/6290904K available (17098K kernel code, 2721K rwdata, 13520K rodata, 3644K init, 3008K bss, 258300K reserved, 0K cma-reserved)337vm-test-run-nixbot-gitlab> gitlab # [    0.638217] devtmpfs: initialized338vm-test-run-nixbot-gitlab> gitlab # [    0.639473] x86/mm: Memory block size: 128MB339vm-test-run-nixbot-gitlab> gitlab # [    0.643228] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns340vm-test-run-nixbot-gitlab> gitlab # [    0.644265] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)341vm-test-run-nixbot-gitlab> gitlab # [    0.645301] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).342vm-test-run-nixbot-gitlab> gitlab # [    0.646357] pinctrl core: initialized pinctrl subsystem343vm-test-run-nixbot-gitlab> gitlab # [    0.647515] PM: RTC time: 10:43:08, date: 2026-06-29344vm-test-run-nixbot-gitlab> gitlab # [    0.651394] NET: Registered PF_NETLINK/PF_ROUTE protocol family345vm-test-run-nixbot-gitlab> gitlab # [    0.653520] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations346vm-test-run-nixbot-gitlab> gitlab # [    0.654281] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations347vm-test-run-nixbot-gitlab> gitlab # [    0.656300] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations348vm-test-run-nixbot-gitlab> gitlab # [    0.657231] audit: initializing netlink subsys (disabled)349vm-test-run-nixbot-gitlab> gitlab # [    0.658414] audit: type=2000 audit(1782729787.941:1): state=initialized audit_enabled=0 res=1350vm-test-run-nixbot-gitlab> gitlab # [    0.658516] thermal_sys: Registered thermal governor 'fair_share'351vm-test-run-nixbot-gitlab> gitlab # [    0.660219] thermal_sys: Registered thermal governor 'bang_bang'352vm-test-run-nixbot-gitlab> gitlab # [    0.661218] thermal_sys: Registered thermal governor 'step_wise'353vm-test-run-nixbot-gitlab> gitlab # [    0.662218] thermal_sys: Registered thermal governor 'user_space'354vm-test-run-nixbot-gitlab> gitlab # [    0.663220] thermal_sys: Registered thermal governor 'power_allocator'355vm-test-run-nixbot-gitlab> gitlab # [    0.664235] cpuidle: using governor menu356vm-test-run-nixbot-gitlab> gitlab # [    0.667460] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5357vm-test-run-nixbot-gitlab> gitlab # [    0.668569] PCI: Using configuration type 1 for base access358vm-test-run-nixbot-gitlab> gitlab # [    0.669219] PCI: Using configuration type 1 for extended access359vm-test-run-nixbot-gitlab> gitlab # [    0.670376] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.360vm-test-run-nixbot-gitlab> gitlab # [    0.674494] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages361vm-test-run-nixbot-gitlab> gitlab # [    0.676219] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page362vm-test-run-nixbot-gitlab> gitlab # [    0.677219] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages363vm-test-run-nixbot-gitlab> gitlab # [    0.678218] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page364vm-test-run-nixbot-gitlab> gitlab # [    0.682996] ACPI: Added _OSI(Module Device)365vm-test-run-nixbot-gitlab> gitlab # [    0.687232] ACPI: Added _OSI(Processor Device)366vm-test-run-nixbot-gitlab> gitlab # [    0.690227] ACPI: Added _OSI(Processor Aggregator Device)367vm-test-run-nixbot-gitlab> gitlab # [    0.699132] ACPI: 1 ACPI AML tables successfully acquired and loaded368vm-test-run-nixbot-gitlab> gitlab # [    0.708518] ACPI: Interpreter enabled369vm-test-run-nixbot-gitlab> gitlab # [    0.711268] ACPI: PM: (supports S0 S3 S4 S5)370vm-test-run-nixbot-gitlab> gitlab # [    0.714241] ACPI: Using IOAPIC for interrupt routing371vm-test-run-nixbot-gitlab> gitlab # [    0.718626] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug372vm-test-run-nixbot-gitlab> gitlab # [    0.725227] PCI: Using E820 reservations for host bridge windows373vm-test-run-nixbot-gitlab> gitlab # [    0.729872] ACPI: Enabled 2 GPEs in block 00 to 0F374vm-test-run-nixbot-gitlab> gitlab # [    0.745820] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])375vm-test-run-nixbot-gitlab> gitlab # [    0.749235] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]376vm-test-run-nixbot-gitlab> gitlab # [    0.756379] acpiphp: Slot [3] registered377vm-test-run-nixbot-gitlab> gitlab # [    0.758306] acpiphp: Slot [4] registered378vm-test-run-nixbot-gitlab> gitlab # [    0.761341] acpiphp: Slot [5] registered379vm-test-run-nixbot-gitlab> gitlab # [    0.763336] acpiphp: Slot [6] registered380vm-test-run-nixbot-gitlab> gitlab # [    0.766317] acpiphp: Slot [7] registered381vm-test-run-nixbot-gitlab> gitlab # [    0.768306] acpiphp: Slot [8] registered382vm-test-run-nixbot-gitlab> gitlab # [    0.770261] acpiphp: Slot [9] registered383vm-test-run-nixbot-gitlab> gitlab # [    0.771274] acpiphp: Slot [10] registered384vm-test-run-nixbot-gitlab> gitlab # [    0.773280] acpiphp: Slot [11] registered385vm-test-run-nixbot-gitlab> gitlab # [    0.774265] acpiphp: Slot [12] registered386vm-test-run-nixbot-gitlab> gitlab # [    0.775273] acpiphp: Slot [13] registered387vm-test-run-nixbot-gitlab> gitlab # [    0.776252] acpiphp: Slot [14] registered388vm-test-run-nixbot-gitlab> gitlab # [    0.778251] acpiphp: Slot [15] registered389vm-test-run-nixbot-gitlab> gitlab # [    0.779658] acpiphp: Slot [16] registered390vm-test-run-nixbot-gitlab> gitlab # [    0.783313] acpiphp: Slot [17] registered391vm-test-run-nixbot-gitlab> gitlab # [    0.786315] acpiphp: Slot [18] registered392vm-test-run-nixbot-gitlab> gitlab # [    0.789348] acpiphp: Slot [19] registered393vm-test-run-nixbot-gitlab> gitlab # [    0.792316] acpiphp: Slot [20] registered394vm-test-run-nixbot-gitlab> gitlab # [    0.795312] acpiphp: Slot [21] registered395vm-test-run-nixbot-gitlab> gitlab # [    0.799317] acpiphp: Slot [22] registered396vm-test-run-nixbot-gitlab> gitlab # [    0.802356] acpiphp: Slot [23] registered397vm-test-run-nixbot-gitlab> gitlab # [    0.805313] acpiphp: Slot [24] registered398vm-test-run-nixbot-gitlab> gitlab # [    0.808327] acpiphp: Slot [25] registered399vm-test-run-nixbot-gitlab> gitlab # [    0.811311] acpiphp: Slot [26] registered400vm-test-run-nixbot-gitlab> gitlab # [    0.815348] acpiphp: Slot [27] registered401vm-test-run-nixbot-gitlab> gitlab # [    0.818314] acpiphp: Slot [28] registered402vm-test-run-nixbot-gitlab> gitlab # [    0.821378] acpiphp: Slot [29] registered403vm-test-run-nixbot-gitlab> gitlab # [    0.824348] acpiphp: Slot [30] registered404vm-test-run-nixbot-gitlab> gitlab # [    0.827291] acpiphp: Slot [31] registered405vm-test-run-nixbot-gitlab> gitlab # [    0.829336] PCI host bridge to bus 0000:00406vm-test-run-nixbot-gitlab> gitlab # [    0.832238] pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7 window]407vm-test-run-nixbot-gitlab> gitlab # [    0.836226] pci_bus 0000:00: root bus resource [io  0x0d00-0xffff window]408vm-test-run-nixbot-gitlab> gitlab # [    0.839225] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]409vm-test-run-nixbot-gitlab> gitlab # [    0.843226] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]410vm-test-run-nixbot-gitlab> gitlab # [    0.845277] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window]411vm-test-run-nixbot-gitlab> gitlab # [    0.848237] pci_bus 0000:00: root bus resource [bus 00-ff]412vm-test-run-nixbot-gitlab> gitlab # [    0.852514] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint413vm-test-run-nixbot-gitlab> gitlab # [    0.860893] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint414vm-test-run-nixbot-gitlab> gitlab # [    0.869522] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint415vm-test-run-nixbot-gitlab> gitlab # [    0.880406] pci 0000:00:01.1: BAR 4 [io  0xc220-0xc22f]416vm-test-run-nixbot-gitlab> gitlab # [    0.884392] pci 0000:00:01.1: BAR 0 [io  0x01f0-0x01f7]: legacy IDE quirk417vm-test-run-nixbot-gitlab> gitlab # [    0.889228] pci 0000:00:01.1: BAR 1 [io  0x03f6]: legacy IDE quirk418vm-test-run-nixbot-gitlab> gitlab # [    0.893224] pci 0000:00:01.1: BAR 2 [io  0x0170-0x0177]: legacy IDE quirk419vm-test-run-nixbot-gitlab> gitlab # [    0.897224] pci 0000:00:01.1: BAR 3 [io  0x0376]: legacy IDE quirk420vm-test-run-nixbot-gitlab> gitlab # [    0.901851] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint421vm-test-run-nixbot-gitlab> gitlab # [    0.909448] pci 0000:00:01.2: BAR 4 [io  0xc140-0xc15f]422vm-test-run-nixbot-gitlab> gitlab # [    0.912420] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint423vm-test-run-nixbot-gitlab> gitlab # [    0.916207] pci 0000:00:01.3: quirk: [io  0x0600-0x063f] claimed by PIIX4 ACPI424vm-test-run-nixbot-gitlab> gitlab # [    0.921264] pci 0000:00:01.3: quirk: [io  0x0700-0x070f] claimed by PIIX4 SMB425vm-test-run-nixbot-gitlab> gitlab # [    0.927361] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint426vm-test-run-nixbot-gitlab> gitlab # [    0.950381] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]427vm-test-run-nixbot-gitlab> gitlab # [    0.953277] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]428vm-test-run-nixbot-gitlab> gitlab # [    0.957328] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref]429vm-test-run-nixbot-gitlab> gitlab # [    0.961681] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]430vm-test-run-nixbot-gitlab> gitlab # [    0.980277] pci 0000:00:02.0: pci_fixup_video+0x0/0x100 took 18554 usecs431vm-test-run-nixbot-gitlab> gitlab # [    0.983571] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint432vm-test-run-nixbot-gitlab> gitlab # [    0.995264] pci 0000:00:03.0: BAR 0 [io  0xc160-0xc17f]433vm-test-run-nixbot-gitlab> gitlab # [    0.999264] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]434vm-test-run-nixbot-gitlab> gitlab # [    1.004360] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref]435vm-test-run-nixbot-gitlab> gitlab # [    1.010266] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]436vm-test-run-nixbot-gitlab> gitlab # [    1.019760] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint437vm-test-run-nixbot-gitlab> gitlab # [    1.031252] pci 0000:00:04.0: BAR 0 [io  0xc180-0xc19f]438vm-test-run-nixbot-gitlab> gitlab # [    1.034252] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]439vm-test-run-nixbot-gitlab> gitlab # [    1.038328] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref]440vm-test-run-nixbot-gitlab> gitlab # [    1.044477] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint441vm-test-run-nixbot-gitlab> gitlab # [    1.049262] pci 0000:00:05.0: BAR 0 [io  0xc1a0-0xc1bf]442vm-test-run-nixbot-gitlab> gitlab # [    1.051236] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]443vm-test-run-nixbot-gitlab> gitlab # [    1.055375] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref]444vm-test-run-nixbot-gitlab> gitlab # [    1.066930] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint445vm-test-run-nixbot-gitlab> gitlab # [    1.081265] pci 0000:00:06.0: BAR 0 [io  0xc1c0-0xc1df]446vm-test-run-nixbot-gitlab> gitlab # [    1.086263] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]447vm-test-run-nixbot-gitlab> gitlab # [    1.090359] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref]448vm-test-run-nixbot-gitlab> gitlab # [    1.099276] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint449vm-test-run-nixbot-gitlab> gitlab # [    1.110260] pci 0000:00:07.0: BAR 0 [io  0xc000-0xc07f]450vm-test-run-nixbot-gitlab> gitlab # [    1.112237] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]451vm-test-run-nixbot-gitlab> gitlab # [    1.114272] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref]452vm-test-run-nixbot-gitlab> gitlab # [    1.119112] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint453vm-test-run-nixbot-gitlab> gitlab # [    1.132370] pci 0000:00:08.0: BAR 0 [io  0xc080-0xc0ff]454vm-test-run-nixbot-gitlab> gitlab # [    1.136279] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]455vm-test-run-nixbot-gitlab> gitlab # [    1.141358] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref]456vm-test-run-nixbot-gitlab> gitlab # [    1.151528] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint457vm-test-run-nixbot-gitlab> gitlab # [    1.164282] pci 0000:00:09.0: BAR 0 [io  0xc1e0-0xc1ff]458vm-test-run-nixbot-gitlab> gitlab # [    1.168253] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]459vm-test-run-nixbot-gitlab> gitlab # [    1.171329] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref]460vm-test-run-nixbot-gitlab> gitlab # [    1.176251] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref]461vm-test-run-nixbot-gitlab> gitlab # [    1.182450] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint462vm-test-run-nixbot-gitlab> gitlab # [    1.191296] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]463vm-test-run-nixbot-gitlab> gitlab # [    1.195359] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref]464vm-test-run-nixbot-gitlab> gitlab # [    1.206387] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint465vm-test-run-nixbot-gitlab> gitlab # [    1.219280] pci 0000:00:0b.0: BAR 0 [io  0xc100-0xc13f]466vm-test-run-nixbot-gitlab> gitlab # [    1.222253] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]467vm-test-run-nixbot-gitlab> gitlab # [    1.226329] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref]468vm-test-run-nixbot-gitlab> gitlab # [    1.234276] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint469vm-test-run-nixbot-gitlab> gitlab # [    1.241258] pci 0000:00:0c.0: BAR 0 [io  0xc200-0xc21f]470vm-test-run-nixbot-gitlab> gitlab # [    1.242234] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff]471vm-test-run-nixbot-gitlab> gitlab # [    1.244272] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref]472vm-test-run-nixbot-gitlab> gitlab # [    1.263429] ACPI: PCI: Interrupt link LNKA configured for IRQ 10473vm-test-run-nixbot-gitlab> gitlab # [    1.268749] ACPI: PCI: Interrupt link LNKB configured for IRQ 10474vm-test-run-nixbot-gitlab> gitlab # [    1.272713] ACPI: PCI: Interrupt link LNKC configured for IRQ 11475vm-test-run-nixbot-gitlab> gitlab # [    1.277708] ACPI: PCI: Interrupt link LNKD configured for IRQ 11476vm-test-run-nixbot-gitlab> gitlab # [    1.282492] ACPI: PCI: Interrupt link LNKS configured for IRQ 9477vm-test-run-nixbot-gitlab> gitlab # [    1.290608] iommu: Default domain type: Translated478vm-test-run-nixbot-gitlab> gitlab # [    1.294242] iommu: DMA domain TLB invalidation policy: lazy mode479vm-test-run-nixbot-gitlab> gitlab # [    1.296595] ACPI: bus type USB registered480vm-test-run-nixbot-gitlab> gitlab # [    1.298369] usbcore: registered new interface driver usbfs481vm-test-run-nixbot-gitlab> gitlab # [    1.300244] usbcore: registered new interface driver hub482vm-test-run-nixbot-gitlab> gitlab # [    1.301271] usbcore: registered new device driver usb483vm-test-run-nixbot-gitlab> gitlab # [    1.304787] NetLabel: Initializing484vm-test-run-nixbot-gitlab> gitlab # [    1.306218] NetLabel:  domain hash size = 128485vm-test-run-nixbot-gitlab> gitlab # [    1.307218] NetLabel:  protocols = UNLABELED CIPSOv4 CALIPSO486vm-test-run-nixbot-gitlab> gitlab # [    1.309276] NetLabel:  unlabeled traffic allowed by default487vm-test-run-nixbot-gitlab> gitlab # [    1.310232] PCI: Using ACPI for IRQ routing488vm-test-run-nixbot-gitlab> gitlab # [    1.313251] pci 0000:00:02.0: vgaarb: setting as boot VGA device489vm-test-run-nixbot-gitlab> gitlab # [    1.314213] pci 0000:00:02.0: vgaarb: bridge control possible490vm-test-run-nixbot-gitlab> gitlab # [    1.314213] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none491vm-test-run-nixbot-gitlab> gitlab # [    1.319222] vgaarb: loaded492vm-test-run-nixbot-gitlab> gitlab # [    1.320395] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0493vm-test-run-nixbot-gitlab> gitlab # [    1.321218] hpet0: 3 comparators, 64-bit 100.000000 MHz counter494vm-test-run-nixbot-gitlab> gitlab # [    1.327311] clocksource: Switched to clocksource kvm-clock495vm-test-run-nixbot-gitlab> gitlab # [    1.333080] VFS: Disk quotas dquot_6.6.0496vm-test-run-nixbot-gitlab> gitlab # [    1.334536] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)497vm-test-run-nixbot-gitlab> gitlab # [    1.341115] pnp: PnP ACPI init498vm-test-run-nixbot-gitlab> gitlab # [    1.345404] pnp: PnP ACPI: found 6 devices499vm-test-run-nixbot-gitlab> gitlab # [    1.361690] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns500vm-test-run-nixbot-gitlab> gitlab # [    1.368358] clocksource: Switched to clocksource acpi_pm501vm-test-run-nixbot-gitlab> gitlab # [    1.372862] NET: Registered PF_INET protocol family502vm-test-run-nixbot-gitlab> gitlab # [    1.380040] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)503vm-test-run-nixbot-gitlab> gitlab # [    1.412682] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)504vm-test-run-nixbot-gitlab> gitlab # [    1.415324] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)505vm-test-run-nixbot-gitlab> gitlab # [    1.418230] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)506vm-test-run-nixbot-gitlab> gitlab # [    1.428083] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)507vm-test-run-nixbot-gitlab> gitlab # [    1.433950] TCP: Hash tables configured (established 65536 bind 65536)508vm-test-run-nixbot-gitlab> gitlab # [    1.440467] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)509vm-test-run-nixbot-gitlab> gitlab # [    1.447062] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)510vm-test-run-nixbot-gitlab> gitlab # [    1.453065] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)511vm-test-run-nixbot-gitlab> gitlab # [    1.458896] NET: Registered PF_UNIX/PF_LOCAL protocol family512vm-test-run-nixbot-gitlab> gitlab # [    1.462726] NET: Registered PF_XDP protocol family513vm-test-run-nixbot-gitlab> gitlab # [    1.465802] pci_bus 0000:00: resource 4 [io  0x0000-0x0cf7 window]514vm-test-run-nixbot-gitlab> gitlab # [    1.469489] pci_bus 0000:00: resource 5 [io  0x0d00-0xffff window]515vm-test-run-nixbot-gitlab> gitlab # [    1.473099] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]516vm-test-run-nixbot-gitlab> gitlab # [    1.477150] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window]517vm-test-run-nixbot-gitlab> gitlab # [    1.480393] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window]518vm-test-run-nixbot-gitlab> gitlab # [    1.482709] pci 0000:00:01.0: PIIX3: Enabling Passive Release519vm-test-run-nixbot-gitlab> gitlab # [    1.484788] pci 0000:00:00.0: Limiting direct PCI/PCI transfers520vm-test-run-nixbot-gitlab> gitlab # [    1.494036] ACPI: \_SB_.LNKD: Enabled at IRQ 11521vm-test-run-nixbot-gitlab> gitlab # [    1.502207] pci 0000:00:01.2: quirk_usb_early_handoff+0x0/0x7c0 took 12361 usecs522vm-test-run-nixbot-gitlab> gitlab # [    1.508587] PCI: CLS 0 bytes, default 64523vm-test-run-nixbot-gitlab> gitlab # [    1.512242] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)524vm-test-run-nixbot-gitlab> gitlab # [    1.512520] Trying to unpack rootfs image as initramfs...525vm-test-run-nixbot-gitlab> gitlab # [    1.513130] software IO TLB: mapped [mem 0x00000000ba4e0000-0x00000000be4e0000] (64MB)526vm-test-run-nixbot-gitlab> gitlab # [    1.527139] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229840d1729, max_idle_ns: 440795276167 ns527vm-test-run-nixbot-gitlab> gitlab # [    1.582084] Initialise system trusted keyrings528vm-test-run-nixbot-gitlab> gitlab # [    1.586409] workingset: timestamp_bits=40 max_order=21 bucket_order=0529vm-test-run-nixbot-gitlab> gitlab # [    1.613989] Key type asymmetric registered530vm-test-run-nixbot-gitlab> gitlab # [    1.615367] Asymmetric key parser 'x509' registered531vm-test-run-nixbot-gitlab> gitlab # [    1.616973] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)532vm-test-run-nixbot-gitlab> gitlab # [    1.620628] io scheduler mq-deadline registered533vm-test-run-nixbot-gitlab> gitlab # [    1.622053] io scheduler kyber registered534vm-test-run-nixbot-gitlab> gitlab # [    1.628080] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled535vm-test-run-nixbot-gitlab> gitlab # [    1.630459] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A536vm-test-run-nixbot-gitlab> gitlab # [    1.634895] Linux agpgart interface v0.103537vm-test-run-nixbot-gitlab> gitlab # [    1.636319] ACPI: bus type drm_connector registered538vm-test-run-nixbot-gitlab> gitlab # [    1.640344] usbcore: registered new interface driver usbserial_generic539vm-test-run-nixbot-gitlab> gitlab # [    1.642308] usbserial: USB Serial support registered for generic540vm-test-run-nixbot-gitlab> gitlab # [    1.644134] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.541vm-test-run-nixbot-gitlab> gitlab # [    1.644134] Please enable it if your BIOS has the CPPC option.542vm-test-run-nixbot-gitlab> gitlab # [    1.648150] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled543vm-test-run-nixbot-gitlab> gitlab # [    1.650549] drop_monitor: Initializing network drop monitor service544vm-test-run-nixbot-gitlab> gitlab # [    1.652644] NET: Registered PF_INET6 protocol family545vm-test-run-nixbot-gitlab> gitlab # [    1.655323] Segment Routing with IPv6546vm-test-run-nixbot-gitlab> gitlab # [    1.656750] In-situ OAM (IOAM) with IPv6547vm-test-run-nixbot-gitlab> gitlab # [    1.662581] IPI shorthand broadcast: enabled548vm-test-run-nixbot-gitlab> gitlab # [    1.671672] sched_clock: Marking stable (1409030531, 262260537)->(1753478294, -82187226)549vm-test-run-nixbot-gitlab> gitlab # [    1.674680] registered taskstats version 1550vm-test-run-nixbot-gitlab> gitlab # [    1.676375] Loading compiled-in X.509 certificates551vm-test-run-nixbot-gitlab> gitlab # [    1.686850] Demotion targets for Node 0: null552vm-test-run-nixbot-gitlab> gitlab # [    1.688669] Key type .fscrypt registered553vm-test-run-nixbot-gitlab> gitlab # [    1.689935] Key type fscrypt-provisioning registered554vm-test-run-nixbot-gitlab> gitlab # [    1.691600] ima: No TPM chip found, activating TPM-bypass!555vm-test-run-nixbot-gitlab> gitlab # [    1.693297] ima: Allocated hash algorithm: sha1556vm-test-run-nixbot-gitlab> gitlab # [    1.694774] ima: No architecture policies found557vm-test-run-nixbot-gitlab> gitlab # [    1.696558] PM:   Magic number: 14:922:730558vm-test-run-nixbot-gitlab> gitlab # [    1.698878] RAS: Correctable Errors collector initialized.559vm-test-run-nixbot-gitlab> gitlab # [    1.704245] clk: Disabling unused clocks560vm-test-run-nixbot-gitlab> gitlab # [    1.705547] PM: genpd: Disabling unused power domains561vm-test-run-nixbot-gitlab> gitlab # [    1.819480] Freeing initrd memory: 27584K562vm-test-run-nixbot-gitlab> gitlab # [    1.828020] Freeing unused decrypted memory: 2028K563vm-test-run-nixbot-gitlab> gitlab # [    1.836852] Freeing unused kernel image (initmem) memory: 3644K564vm-test-run-nixbot-gitlab> gitlab # [    1.841661] Write protecting the kernel read-only data: 32768k565vm-test-run-nixbot-gitlab> gitlab # [    1.849114] Freeing unused kernel image (text/rodata gap) memory: 1332K566vm-test-run-nixbot-gitlab> gitlab # [    1.855704] Freeing unused kernel image (rodata/data gap) memory: 816K567vm-test-run-nixbot-gitlab> gitlab # [    1.919081] x86/mm: Checked W+X mappings: passed, no W+X pages found.568vm-test-run-nixbot-gitlab> gitlab # [    1.921125] Run /init as init process569vm-test-run-nixbot-gitlab> gitlab # [    1.936902] systemd[1]: Inserted module 'autofs4'570vm-test-run-nixbot-gitlab> gitlab # [    1.965725] fuse: init (API version 7.45)571vm-test-run-nixbot-gitlab> gitlab # [    1.976848] ACPI: \_SB_.LNKC: Enabled at IRQ 10572vm-test-run-nixbot-gitlab> gitlab # [    1.998905] ACPI: \_SB_.LNKA: Enabled at IRQ 10573vm-test-run-nixbot-gitlab> gitlab # [    2.004150] ACPI: \_SB_.LNKB: Enabled at IRQ 11574vm-test-run-nixbot-gitlab> gitlab # [    2.054380] systemd[1]: Successfully made /usr/ read-only.575vm-test-run-nixbot-gitlab> gitlab # [    2.392311] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)576vm-test-run-nixbot-gitlab> gitlab # [    2.400959] systemd[1]: Detected virtualization kvm.577vm-test-run-nixbot-gitlab> gitlab # [    2.402510] systemd[1]: Detected architecture x86-64.578vm-test-run-nixbot-gitlab> gitlab # [    2.404102] systemd[1]: Running in initrd.579vm-test-run-nixbot-gitlab> gitlab # [    2.405892] systemd[1]: Initializing machine ID from random generator.580vm-test-run-nixbot-gitlab> gitlab # [    2.407960] systemd[1]: Hostname set to <gitlab>.581vm-test-run-nixbot-gitlab> gitlab # [    2.494448] systemd[1]: Queued start job for default target Initrd Default Target.582vm-test-run-nixbot-gitlab> gitlab # [    2.681176] systemd[1]: Created slice Slice /system/modprobe.583vm-test-run-nixbot-gitlab> gitlab # [    2.686458] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.584vm-test-run-nixbot-gitlab> gitlab # [    2.692709] systemd[1]: Expecting device /dev/disk/by-label/nix-store...585vm-test-run-nixbot-gitlab> gitlab # [    2.697845] systemd[1]: Expecting device /dev/disk/by-label/nixos...586vm-test-run-nixbot-gitlab> gitlab # [    2.702873] systemd[1]: Reached target Path Units.587vm-test-run-nixbot-gitlab> gitlab # [    2.706849] systemd[1]: Reached target Slice Units.588vm-test-run-nixbot-gitlab> gitlab # [    2.710839] systemd[1]: Reached target Swaps.589vm-test-run-nixbot-gitlab> gitlab # [    2.714537] systemd[1]: Reached target Timer Units.590vm-test-run-nixbot-gitlab> gitlab # [    2.718842] systemd[1]: Listening on D-Bus System Message Bus Socket.591vm-test-run-nixbot-gitlab> gitlab # [    2.723066] systemd[1]: Listening on Journal Socket (/dev/log).592vm-test-run-nixbot-gitlab> gitlab # [    2.727112] systemd[1]: Listening on Journal Sockets.593vm-test-run-nixbot-gitlab> gitlab # [    2.730748] systemd[1]: Listening on udev Control Socket.594vm-test-run-nixbot-gitlab> gitlab # [    2.734506] systemd[1]: Listening on udev Kernel Socket.595vm-test-run-nixbot-gitlab> gitlab # [    2.736652] systemd[1]: Reached target Socket Units.596vm-test-run-nixbot-gitlab> gitlab # [    2.743010] systemd[1]: Starting Create List of Static Device Nodes...597vm-test-run-nixbot-gitlab> gitlab # [    2.749703] systemd[1]: Starting Load Kernel Module 9pnet_virtio...598vm-test-run-nixbot-gitlab> gitlab # [    2.770421] systemd[1]: Starting Load Kernel Module configfs...599vm-test-run-nixbot-gitlab> gitlab # [    2.775773] systemd[1]: Starting Journal Service...600vm-test-run-nixbot-gitlab> gitlab # [    2.792845] systemd[1]: Starting Load Kernel Modules...601vm-test-run-nixbot-gitlab> gitlab # [    2.797167] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki602vm-test-run-nixbot-gitlab> gitlab # [    2.802910] systemd-journald[153]: Collecting audit messages is disabled.603vm-test-run-nixbot-gitlab> gitlab # [    2.813805] systemd[1]: Starting Coldplug All udev Devices...604vm-test-run-nixbot-gitlab> gitlab # [    2.832813] netfs: FS-Cache loaded605vm-test-run-nixbot-gitlab> gitlab # [    2.840675] 9pnet: Installing 9P2000 support606vm-test-run-nixbot-gitlab> gitlab # [    2.845397] systemd[1]: Finished Create List of Static Device Nodes.607vm-test-run-nixbot-gitlab> gitlab # [    2.851242] systemd[1]: modprobe@configfs.service: Deactivated successfully.608vm-test-run-nixbot-gitlab> gitlab # [    2.857090] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.609vm-test-run-nixbot-gitlab> gitlab # [    2.861233] systemd[1]: Finished Load Kernel Module configfs.610vm-test-run-nixbot-gitlab> gitlab # [    2.863320] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev611vm-test-run-nixbot-gitlab> gitlab # [    2.867393] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.612vm-test-run-nixbot-gitlab> gitlab # [    2.871471] systemd[1]: Finished Load Kernel Module 9pnet_virtio.613vm-test-run-nixbot-gitlab> gitlab # [    2.877665] systemd[1]: Finished Load Kernel Modules.614vm-test-run-nixbot-gitlab> gitlab # [    2.880242] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config615vm-test-run-nixbot-gitlab> gitlab # [    2.886085] systemd[1]: Starting Apply Kernel Variables...616vm-test-run-nixbot-gitlab> gitlab # [    2.631142] systemd-modules-load[156]: Using 4 probe threads617vm-test-run-nixbot-gitlab> gitlab # [    2.633279] systemd-modules-load[156]: Inserted module 'virtio_balloon'618vm-test-run-nixbot-gitlab> gitlab # [    2.635178] systemd-modules-load[156]: Inserted module 'virtio_gpu'619vm-test-run-nixbot-gitlab> gitlab # [    2.637041] systemd-modules-load[156]: Inserted module 'dm_mod'620vm-test-run-nixbot-gitlab> gitlab # [    2.900435] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...621vm-test-run-nixbot-gitlab> gitlab # [    2.902857] systemd[1]: Started Journal Service.622vm-test-run-nixbot-gitlab> gitlab # [    2.644975] systemd[1]: Finished Apply Kernel Variables.623vm-test-run-nixbot-gitlab> gitlab # [    2.651895] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.624vm-test-run-nixbot-gitlab> gitlab # [    2.656540] systemd[1]: Starting Create Static Device Nodes in /dev...625vm-test-run-nixbot-gitlab> gitlab # [    2.678392] systemd[1]: Finished Create Static Device Nodes in /dev.626vm-test-run-nixbot-gitlab> gitlab # [    2.680161] systemd[1]: Reached target Preparation for Local File Systems.627vm-test-run-nixbot-gitlab> gitlab # [    2.681794] systemd[1]: Reached target Local File Systems.628vm-test-run-nixbot-gitlab> gitlab # [    2.683639] systemd[1]: Starting Create System Files and Directories...629vm-test-run-nixbot-gitlab> gitlab # [    2.688090] systemd[1]: Starting Rule-based Manager for Device Events and Files...630vm-test-run-nixbot-gitlab> gitlab # [    2.698589] systemd[1]: Finished Create System Files and Directories.631vm-test-run-nixbot-gitlab> gitlab # [    2.708798] systemd-udevd[197]: Using default interface naming scheme 'v260'.632vm-test-run-nixbot-gitlab> gitlab # [    2.725385] systemd[1]: Started Rule-based Manager for Device Events and Files.633vm-test-run-nixbot-gitlab> gitlab # [    2.739639] systemd[1]: Finished Coldplug All udev Devices.634vm-test-run-nixbot-gitlab> gitlab # [    2.741209] systemd[1]: Reached target System Initialization.635vm-test-run-nixbot-gitlab> gitlab # [    2.742729] systemd[1]: Reached target Basic System.636vm-test-run-nixbot-gitlab> gitlab # [    3.158497] uhci_hcd 0000:00:01.2: UHCI Host Controller637vm-test-run-nixbot-gitlab> gitlab # [    3.159960] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1638vm-test-run-nixbot-gitlab> gitlab # [    3.162827] uhci_hcd 0000:00:01.2: detected 2 ports639vm-test-run-nixbot-gitlab> gitlab # [    3.171798] SCSI subsystem initialized640vm-test-run-nixbot-gitlab> gitlab # [    3.175651] virtio_blk virtio4: 4/0/0 default/read/poll queues641vm-test-run-nixbot-gitlab> gitlab # [    3.179241] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140642vm-test-run-nixbot-gitlab> gitlab # [    3.181311] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18643vm-test-run-nixbot-gitlab> gitlab # [    3.183304] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1644vm-test-run-nixbot-gitlab> gitlab # [    3.185049] usb usb1: Product: UHCI Host Controller645vm-test-run-nixbot-gitlab> gitlab # [    3.186442] usb usb1: Manufacturer: Linux 6.18.34 uhci_hcd646vm-test-run-nixbot-gitlab> gitlab # [    3.187859] usb usb1: SerialNumber: 0000:00:01.2647vm-test-run-nixbot-gitlab> gitlab # [    3.188451] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)648vm-test-run-nixbot-gitlab> gitlab # [    3.198231] hub 1-0:1.0: USB hub found649vm-test-run-nixbot-gitlab> gitlab # [    3.198808] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12650vm-test-run-nixbot-gitlab> gitlab # [    3.199305] hub 1-0:1.0: 2 ports detected651vm-test-run-nixbot-gitlab> gitlab # [    3.201795] virtio_blk virtio5: 4/0/0 default/read/poll queues652vm-test-run-nixbot-gitlab> gitlab # [    3.203202] serio: i8042 KBD port at 0x60,0x64 irq 1653vm-test-run-nixbot-gitlab> gitlab # [    3.205898] serio: i8042 AUX port at 0x60,0x64 irq 12654vm-test-run-nixbot-gitlab> gitlab # [    3.211073] scsi host0: ata_piix655vm-test-run-nixbot-gitlab> gitlab # [    3.213074] virtio_blk virtio5: [vdb] 11498048 512-byte logical blocks (5.89 GB/5.48 GiB)656vm-test-run-nixbot-gitlab> gitlab # [    3.213440] scsi host1: ata_piix657vm-test-run-nixbot-gitlab> gitlab # [    2.991193] (udev-worker)[210]: Network interface NamePolicy= disabled on kernel command line.658vm-test-run-nixbot-gitlab> gitlab # [    3.262145] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0659vm-test-run-nixbot-gitlab> gitlab # [    3.264142] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0660vm-test-run-nixbot-gitlab> gitlab # [    3.004215] systemd[1]: Starting Virtual Console Setup...661vm-test-run-nixbot-gitlab> gitlab # [    3.008476] (udev-worker)[206]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.662vm-test-run-nixbot-gitlab> gitlab # [    3.013235] (udev-worker)[206]: Network interface NamePolicy= disabled on kernel command line.663vm-test-run-nixbot-gitlab> gitlab # [    3.024155] systemd[1]: Found device /dev/disk/by-label/nixos.664vm-test-run-nixbot-gitlab> gitlab # [    3.026252] systemd[1]: Reached target Initrd Root Device.665vm-test-run-nixbot-gitlab> gitlab # [    3.027912] systemd-vconsole-setup[236]: Configuration of first virtual console was skipped, ignoring remaining ones.666vm-test-run-nixbot-gitlab> gitlab # [    3.031512] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...667vm-test-run-nixbot-gitlab> gitlab # [    3.035445] systemd[1]: Finished Virtual Console Setup.668vm-test-run-nixbot-gitlab> gitlab # [    3.038446] systemd[1]: Found device /dev/disk/by-label/nix-store.669vm-test-run-nixbot-gitlab> gitlab # [    3.060777] systemd-fsck[244]: nixos: clean, 12/65536 files, 13019/262144 blocks670vm-test-run-nixbot-gitlab> gitlab # [    3.077069] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.671vm-test-run-nixbot-gitlab> gitlab # [    3.421719] ata2: found unknown device (class 0)672vm-test-run-nixbot-gitlab> gitlab # [    3.422416] usb 1-1: new full-speed USB device number 2 using uhci_hcd673vm-test-run-nixbot-gitlab> gitlab # [    3.426996] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100674vm-test-run-nixbot-gitlab> gitlab # [    3.435921] scsi 1:0:0:0: CD-ROM            QEMU     QEMU DVD-ROM     2.5+ PQ: 0 ANSI: 5675vm-test-run-nixbot-gitlab> gitlab # [    3.478020] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0676vm-test-run-nixbot-gitlab> gitlab # [    3.528552] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray677vm-test-run-nixbot-gitlab> gitlab # [    3.564799] cdrom: Uniform CD-ROM driver Revision: 3.20678vm-test-run-nixbot-gitlab> gitlab # [    3.599030] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00679vm-test-run-nixbot-gitlab> gitlab # [    3.603977] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10680vm-test-run-nixbot-gitlab> gitlab # [    3.608366] usb 1-1: Product: QEMU USB Tablet681vm-test-run-nixbot-gitlab> gitlab # [    3.611147] usb 1-1: Manufacturer: QEMU682vm-test-run-nixbot-gitlab> gitlab # [    3.613745] usb 1-1: SerialNumber: 28754-0000:00:01.2-1683vm-test-run-nixbot-gitlab> gitlab # [    3.656354] hid: raw HID events driver (C) Jiri Kosina684vm-test-run-nixbot-gitlab> gitlab # [    3.668963] usbcore: registered new interface driver usbhid685vm-test-run-nixbot-gitlab> gitlab # [    3.672414] usbhid: USB HID core driver686vm-test-run-nixbot-gitlab> gitlab # [    3.685640] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2687vm-test-run-nixbot-gitlab> gitlab # [    3.690964] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0688vm-test-run-nixbot-gitlab> gitlab # [    3.583716] systemd[1]: Mounting /sysroot...689vm-test-run-nixbot-gitlab> gitlab # [    4.021804] EXT4-fs (vda): mounted filesystem 007f89d7-2247-44f5-8ffd-793483f8687d r/w with ordered data mode. Quota mode: none.690vm-test-run-nixbot-gitlab> gitlab # [    3.764163] systemd[1]: Mounted /sysroot.691vm-test-run-nixbot-gitlab> gitlab # [    3.765950] systemd[1]: Reached target Initrd Root File System.692vm-test-run-nixbot-gitlab> gitlab # [    3.769492] systemd[1]: Mounting /sysroot/nix/.ro-store...693vm-test-run-nixbot-gitlab> gitlab # [    3.772682] systemd[1]: Mounting /sysroot/nix/.rw-store...694vm-test-run-nixbot-gitlab> gitlab # [    3.776909] systemd[1]: Mounting /sysroot/run...695vm-test-run-nixbot-gitlab> gitlab # [    3.792376] systemd[1]: Mounting /sysroot/tmp/shared...696vm-test-run-nixbot-gitlab> gitlab # [    3.795824] systemd[1]: Mounting /sysroot/tmp/xchg...697vm-test-run-nixbot-gitlab> gitlab # [    3.801814] systemd[1]: Start[    4.064306] erofs (device vdb): mounted with root inode @ nid 128.698vm-test-run-nixbot-gitlab> gitlab # ing Mountpoints Configured in the Real Root...699vm-test-run-nixbot-gitlab> gitlab # [    3.810360] systemd[1]: Mounted /sysroot/nix/.ro-store.700vm-test-run-nixbot-gitlab> gitlab # [    3.813495] systemd[1]: Mounted /sysroot/nix/.rw-store.701vm-test-run-nixbot-gitlab> gitlab # [    3.814797] systemd[1]: Mounted /sysroot/run.702vm-test-run-nixbot-gitlab> gitlab # [    3.818230] systemd[1]: Start[    4.080381] 9p: Installing v9fs 9p2000 file system support703vm-test-run-nixbot-gitlab> gitlab # ing rw-sysroot-nix-store.service...704vm-test-run-nixbot-gitlab> gitlab # [    3.821329] systemd-sysroot-fstab-check[272]: /sysroot should be mounted in the initrd, will request daemon-reload.705vm-test-run-nixbot-gitlab> gitlab # [    3.824283] systemd[1]: Reload requested from client PID 272 ('systemd-sysroot') (unit initrd-parse-etc.service)...706vm-test-run-nixbot-gitlab> gitlab # [    3.827179] systemd[1]: Reloading...707vm-test-run-nixbot-gitlab> gitlab # [    4.034894] systemd[1]: Reloading finished in 211 ms.708vm-test-run-nixbot-gitlab> gitlab # [    4.092206] systemd-sysroot-fstab-check[272]: Requesting initrd-fs.target/start/replace...709vm-test-run-nixbot-gitlab> gitlab # [    4.237246] systemd[1]: Mounted /sysroot/tmp/shared.710vm-test-run-nixbot-gitlab> gitlab # [    4.240924] systemd[1]: Mounted /sysroot/tmp/xchg.711vm-test-run-nixbot-gitlab> gitlab # [    4.244570] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.712vm-test-run-nixbot-gitlab> gitlab # [    4.249270] systemd[1]: Finished rw-sysroot-nix-store.service.713vm-test-run-nixbot-gitlab> gitlab # [    4.257422] systemd-sysroot-fstab-check[272]: Requesting swap.target/start/replace...714vm-test-run-nixbot-gitlab> gitlab # [    4.265473] systemd[1]: Starting rw-sysroot-nix-store.service...715vm-test-run-nixbot-gitlab> gitlab # [    4.268672] systemd[1]: initrd-parse-etc.service: Deactivated successfully.716vm-test-run-nixbot-gitlab> gitlab # [    4.273140] systemd[1]: Finished Mountpoints Configured in the Real Root.717vm-test-run-nixbot-gitlab> gitlab # [    4.276537] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.718vm-test-run-nixbot-gitlab> gitlab # [    4.291732] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.719vm-test-run-nixbot-gitlab> gitlab # [    4.299524] systemd[1]: Finished rw-sysroot-nix-store.service.720vm-test-run-nixbot-gitlab> gitlab # [    4.584685] systemd[1]: Mounting /sysroot/nix/store...721vm-test-run-nixbot-gitlab> gitlab # [    4.647114] systemd[1]: Mounted /sysroot/nix/store.722vm-test-run-nixbot-gitlab> gitlab # [    4.650786] systemd[1]: Reached target Initrd File Systems.723vm-test-run-nixbot-gitlab> gitlab # [    4.658598] systemd[1]: Starting Find NixOS closure...724vm-test-run-nixbot-gitlab> gitlab # [    4.666521] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...725vm-test-run-nixbot-gitlab> gitlab # [    4.702992] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.726vm-test-run-nixbot-gitlab> gitlab # [    4.708192] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.727vm-test-run-nixbot-gitlab> gitlab # [    4.723388] systemd[1]: Finished Find NixOS closure.728vm-test-run-nixbot-gitlab> gitlab # [    4.729179] systemd[1]: Reached target Initrd Default Target.729vm-test-run-nixbot-gitlab> gitlab # [    4.736819] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...730vm-test-run-nixbot-gitlab> gitlab # [    4.772187] systemd[1]: Stopped target Initrd Default Target.731vm-test-run-nixbot-gitlab> gitlab # [    4.775753] systemd[1]: Stopped target Basic System.732vm-test-run-nixbot-gitlab> gitlab # [    4.778926] systemd[1]: Stopped target Initrd Root Device.733vm-test-run-nixbot-gitlab> gitlab # [    4.782338] systemd[1]: Stopped target Path Units.734vm-test-run-nixbot-gitlab> gitlab # [    4.785423] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.735vm-test-run-nixbot-gitlab> gitlab # [    4.789908] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.736vm-test-run-nixbot-gitlab> gitlab # [    4.794438] systemd[1]: Stopped target Slice Units.737vm-test-run-nixbot-gitlab> gitlab # [    4.797598] systemd[1]: Stopped target Socket Units.738vm-test-run-nixbot-gitlab> gitlab # [    4.800624] systemd[1]: Stopped target System Initialization.739vm-test-run-nixbot-gitlab> gitlab # [    4.803926] systemd[1]: Stopped target Swaps.740vm-test-run-nixbot-gitlab> gitlab # [    4.806563] systemd[1]: Stopped target Timer Units.741vm-test-run-nixbot-gitlab> gitlab # [    4.809428] systemd[1]: dbus.socket: Deactivated successfully.742vm-test-run-nixbot-gitlab> gitlab # [    4.812628] systemd[1]: Closed D-Bus System Message Bus Socket.743vm-test-run-nixbot-gitlab> gitlab # [    4.815465] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.744vm-test-run-nixbot-gitlab> gitlab # [    4.818880] systemd[1]: Stopped Find NixOS closure.745vm-test-run-nixbot-gitlab> gitlab # [    4.821304] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio746vm-test-run-nixbot-gitlab> gitlab # [    4.826915] systemd[1]: Starting rw-sysroot-nix-store.service...747vm-test-run-nixbot-gitlab> gitlab # [    4.830492] systemd[1]: systemd-sysctl.service: Deactivated successfully.748vm-test-run-nixbot-gitlab> gitlab # [    4.833668] systemd[1]: Stopped Apply Kernel Variables.749vm-test-run-nixbot-gitlab> gitlab # [    4.835604] systemd[1]: systemd-modules-load.service: Deactivated successfully.750vm-test-run-nixbot-gitlab> gitlab # [    4.837901] systemd[1]: Stopped Load Kernel Modules.751vm-test-run-nixbot-gitlab> gitlab # [    4.842109] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.752vm-test-run-nixbot-gitlab> gitlab # [    4.844997] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.753vm-test-run-nixbot-gitlab> gitlab # [    4.849766] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.754vm-test-run-nixbot-gitlab> gitlab # [    4.852316] systemd[1]: Stopped Create System Files and Directories.755vm-test-run-nixbot-gitlab> gitlab # [    4.854571] systemd[1]: Stopped target Local File Systems.756vm-test-run-nixbot-gitlab> gitlab # [    4.856718] systemd[1]: Stopped target Preparation for Local File Systems.757vm-test-run-nixbot-gitlab> gitlab # [    4.859308] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.758vm-test-run-nixbot-gitlab> gitlab # [    4.861861] systemd[1]: Stopped Coldplug All udev Devices.759vm-test-run-nixbot-gitlab> gitlab # [    4.863954] systemd[1]: Stopping Rule-based Manager for Device Events and Files...760vm-test-run-nixbot-gitlab> gitlab # [    4.866608] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.761vm-test-run-nixbot-gitlab> gitlab # [    4.869774] systemd[1]: Stopped Virtual Console Setup.762vm-test-run-nixbot-gitlab> gitlab # [    4.873758] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.763vm-test-run-nixbot-gitlab> gitlab # [    4.877514] systemd[1]: Finished rw-sysroot-nix-store.service.764vm-test-run-nixbot-gitlab> gitlab # [    4.881474] systemd[1]: systemd-udevd.service: Deactivated successfully.765vm-test-run-nixbot-gitlab> gitlab # [    4.884088] systemd[1]: Stopped Rule-based Manager for Device Events and Files.766vm-test-run-nixbot-gitlab> gitlab # [    4.886373] systemd[1]: systemd-udevd.service: Consumed 1.113s CPU time over 2.193s wall clock time, 25.8M memory peak.767vm-test-run-nixbot-gitlab> gitlab # [    4.893659] systemd[1]: initrd-cleanup.service: Deactivated successfully.768vm-test-run-nixbot-gitlab> gitlab # [    4.896233] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.769vm-test-run-nixbot-gitlab> gitlab # [    4.902309] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.770vm-test-run-nixbot-gitlab> gitlab # [    4.904563] systemd[1]: Closed udev Control Socket.771vm-test-run-nixbot-gitlab> gitlab # [    4.909473] systemd[1]: Starting Cleanup udev Database...772vm-test-run-nixbot-gitlab> gitlab # [    4.911401] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.773vm-test-run-nixbot-gitlab> gitlab # [    4.913644] systemd[1]: Stopped Create Static Device Nodes in /dev.774vm-test-run-nixbot-gitlab> gitlab # [    4.916910] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.775vm-test-run-nixbot-gitlab> gitlab # [    4.919390] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.776vm-test-run-nixbot-gitlab> gitlab # [    4.922794] systemd[1]: kmod-static-nodes.service: Deactivated successfully.777vm-test-run-nixbot-gitlab> gitlab # [    4.924975] systemd[1]: Stopped Create List of Static Device Nodes.778vm-test-run-nixbot-gitlab> gitlab # [    4.931520] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.779vm-test-run-nixbot-gitlab> gitlab # [    4.933840] systemd[1]: Finished Cleanup udev Database.780vm-test-run-nixbot-gitlab> gitlab # [    4.937651] systemd[1]: Reached target Switch Root.781vm-test-run-nixbot-gitlab> gitlab # [    4.939755] systemd[1]: Starting NixOS Activation...782vm-test-run-nixbot-gitlab> gitlab # [    5.047480] initrd-nixos-activation-start[573]: booting system configuration /nix/store/yy7030swsiwlxjxgzf612g3r62cyp71p-nixos-system-gitlab-test783vm-test-run-nixbot-gitlab> gitlab # [    5.094634] initrd-nixos-activation-start[573]: running activation script...784vm-test-run-nixbot-gitlab> gitlab # [    5.324515] initrd-nixos-activation-start[596]: setting up /etc...785vm-test-run-nixbot-gitlab> gitlab # [    5.442460] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.786vm-test-run-nixbot-gitlab> gitlab # [    5.446933] systemd[1]: Finished NixOS Activation.787vm-test-run-nixbot-gitlab> gitlab # [    5.453633] systemd[1]: Starting Switch Root...788vm-test-run-nixbot-gitlab> gitlab # [    5.488271] systemd[1]: Switching root.789vm-test-run-nixbot-gitlab> gitlab # [    6.119327] systemd-journald[153]: Received SIGTERM from PID 1 (systemd).790vm-test-run-nixbot-gitlab> gitlab # [    6.243584] NET: Registered PF_VSOCK protocol family791vm-test-run-nixbot-gitlab> gitlab # [    6.613119] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)792vm-test-run-nixbot-gitlab> gitlab # [    6.629711] systemd[1]: Detected virtualization kvm.793vm-test-run-nixbot-gitlab> gitlab # [    6.632898] systemd[1]: Detected architecture x86-64.794vm-test-run-nixbot-gitlab> gitlab # [    6.636206] systemd[1]: Detected first boot.795vm-test-run-nixbot-gitlab> gitlab # [    6.640402] systemd[1]: Initializing machine ID from random generator.796vm-test-run-nixbot-gitlab> gitlab # [    6.783410] systemd[1]: bpf-restrict-fs: LSM BPF program attached797vm-test-run-nixbot-gitlab> gitlab # [    6.855015] systemd[1]: Applying preset policy.798vm-test-run-nixbot-gitlab> gitlab # [    7.013534] systemd[1]: Populated /etc with preset unit settings.799vm-test-run-nixbot-gitlab> gitlab # [    7.130798] systemd[1]: initrd-switch-root.service: Deactivated successfully.800vm-test-run-nixbot-gitlab> gitlab # [    7.133077] systemd[1]: Stopped initrd-switch-root.service.801vm-test-run-nixbot-gitlab> gitlab # [    7.136582] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.802vm-test-run-nixbot-gitlab> gitlab # [    7.139772] systemd[1]: Created slice Slice /system/getty.803vm-test-run-nixbot-gitlab> gitlab # [    7.141952] systemd[1]: Created slice GitLab DevOps Platform Slice.804vm-test-run-nixbot-gitlab> gitlab # [    7.144292] systemd[1]: Created slice User and Session Slice.805vm-test-run-nixbot-gitlab> gitlab # [    7.145933] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.806vm-test-run-nixbot-gitlab> gitlab # [    7.147999] systemd[1]: Started Forward Password Requests to Wall Directory Watch.807vm-test-run-nixbot-gitlab> gitlab # [    7.149900] systemd[1]: Expecting device /dev/hvc0...808vm-test-run-nixbot-gitlab> gitlab # [    7.151175] systemd[1]: Expecting device /dev/ttyS0...809vm-test-run-nixbot-gitlab> gitlab # [    7.152568] systemd[1]: Reached target Local Encrypted Volumes.810vm-test-run-nixbot-gitlab> gitlab # [    7.154124] systemd[1]: Stopped target initrd-fs.target.811vm-test-run-nixbot-gitlab> gitlab # [    7.155528] systemd[1]: Stopped target initrd-root-fs.target.812vm-test-run-nixbot-gitlab> gitlab # [    7.156967] systemd[1]: Stopped target initrd-switch-root.target.813vm-test-run-nixbot-gitlab> gitlab # [    7.158546] systemd[1]: Reached target Virtual Machines and Containers.814vm-test-run-nixbot-gitlab> gitlab # [    7.160189] systemd[1]: Reached target Path Units.815vm-test-run-nixbot-gitlab> gitlab # [    7.161509] systemd[1]: Reached target Remote File Systems.816vm-test-run-nixbot-gitlab> gitlab # [    7.162947] systemd[1]: Reached target Slice Units.817vm-test-run-nixbot-gitlab> gitlab # [    7.164321] systemd[1]: Reached target Swaps.818vm-test-run-nixbot-gitlab> gitlab # [    7.166618] systemd[1]: Listening on Process Core Dump Socket.819vm-test-run-nixbot-gitlab> gitlab # [    7.168758] systemd[1]: Listening on Credential Encryption/Decryption.820vm-test-run-nixbot-gitlab> gitlab # [    7.172030] systemd[1]: Starting Journal Log Access Socket...821vm-test-run-nixbot-gitlab> gitlab # [    7.174220] systemd[1]: Listening on Journal Audit Socket.822vm-test-run-nixbot-gitlab> gitlab # [    7.175941] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.823vm-test-run-nixbot-gitlab> gitlab # [    7.177897] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki824vm-test-run-nixbot-gitlab> gitlab # [    7.180447] systemd[1]: Listening on udev Control Socket.825vm-test-run-nixbot-gitlab> gitlab # [    7.186630] systemd[1]: Mounting Huge Pages File System...826vm-test-run-nixbot-gitlab> gitlab # [    7.189572] systemd[1]: Mounting POSIX Message Queue File System...827vm-test-run-nixbot-gitlab> gitlab # [    7.192645] systemd[1]: Mounting Kernel Debug File System...828vm-test-run-nixbot-gitlab> gitlab # [    7.196388] systemd[1]: Mounting Kernel Trace File System...829vm-test-run-nixbot-gitlab> gitlab # [    7.204748] systemd[1]: Starting Create List of Static Device Nodes...830vm-test-run-nixbot-gitlab> gitlab # [    7.206569] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio831vm-test-run-nixbot-gitlab> gitlab # [    7.210968] systemd[1]: Starting Load Kernel Module configfs...832vm-test-run-nixbot-gitlab> gitlab # [    7.212552] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm833vm-test-run-nixbot-gitlab> gitlab # [    7.212635] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore834vm-test-run-nixbot-gitlab> gitlab # [    7.212710] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse835vm-test-run-nixbot-gitlab> gitlab # [    7.218114] systemd[1]: Mounting FUSE Control File System...836vm-test-run-nixbot-gitlab> gitlab # [    7.228998] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67837vm-test-run-nixbot-gitlab> gitlab # [    7.252367] systemd[1]: Starting Journal Service...838vm-test-run-nixbot-gitlab> gitlab # [    7.284953] systemd[1]: Starting Load Kernel Modules...839vm-test-run-nixbot-gitlab> gitlab # [    7.288292] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...840vm-test-run-nixbot-gitlab> gitlab # [    7.291121] systemd[1]: Starting Remount Root and Kernel File Systems...841vm-test-run-nixbot-gitlab> gitlab # [    7.292889] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki842vm-test-run-nixbot-gitlab> gitlab # [    7.297783] systemd-journald[792]: Collecting audit messages is enabled.843vm-test-run-nixbot-gitlab> gitlab # [    7.300172] systemd[1]: Starting Coldplug All udev Devices...844vm-test-run-nixbot-gitlab> gitlab # [    7.304872] systemd[1]: Listening on Journal Log Access Socket.845vm-test-run-nixbot-gitlab> gitlab # [    7.305104] systemd[1]: Mounted Huge Pages File System.846vm-test-run-nixbot-gitlab> gitlab # [    7.307626] systemd[1]: Mounted POSIX Message Queue File System.847vm-test-run-nixbot-gitlab> gitlab # [    7.309813] systemd[1]: Mounted Kernel Debug File System.848vm-test-run-nixbot-gitlab> gitlab # [    7.311394] loop: module loaded849vm-test-run-nixbot-gitlab> gitlab # [    7.311454] systemd[1]: Mounted Kernel Trace File System.850vm-test-run-nixbot-gitlab> gitlab # [    7.315022] EXT4-fs (vda): re-mounted 007f89d7-2247-44f5-8ffd-793483f8687d.851vm-test-run-nixbot-gitlab> gitlab # [    7.317983] systemd[1]: Finished Create List of Static Device Nodes.852vm-test-run-nixbot-gitlab> gitlab # [    7.058380] systemd[1]: Queued start job for [    7.320779] systemd[1]: Started Journal Service.853vm-test-run-nixbot-gitlab> gitlab # default target Multi-User System.854vm-test-run-nixbot-gitlab> gitlab # [    7.061909] systemd[1]: Unnecessary job was removed for /dev/vdb.855vm-test-run-nixbot-gitlab> gitlab # [    7.064292] systemd[1]: systemd-journald.service: Deactivated successfully.856vm-test-run-nixbot-gitlab> gitlab # [    7.066556] systemd-modules-load[794]: Using 3 probe threads857vm-test-run-nixbot-gitlab> gitlab # [    7.068451] systemd-modules-load[794]: Inserted module 'loop'858vm-test-run-nixbot-gitlab> gitlab # [    7.070408] systemd-modules-load[794]: Inserted module 'tls'859vm-test-run-nixbot-gitlab> gitlab # [    7.072209] systemd[1]: modprobe@configfs.service: Deactivated successfully.860vm-test-run-nixbot-gitlab> gitlab # [    7.074430] systemd[1]: Finished Load Kernel Module configfs.861vm-test-run-nixbot-gitlab> gitlab # [    7.076328] systemd[1]: Mounted FUSE Control File System.862vm-test-run-nixbot-gitlab> gitlab # [    7.078186] systemd[1]: Finished Load Kernel Modules.863vm-test-run-nixbot-gitlab> gitlab # [    7.079855] systemd[1]: Finished Remount Root and Kernel File Systems.864vm-test-run-nixbot-gitlab> gitlab # [    7.081830] systemd[1]: Mounting Kernel Configuration File System...865vm-test-run-nixbot-gitlab> gitlab # [    7.083586] systemd[1]: Starting Firewall...866vm-test-run-nixbot-gitlab> gitlab # [    7.084861] systemd[1]: Starting Flush Journal to Persistent Storage...867vm-test-run-nixbot-gitlab> gitlab # [    7.086866] systemd-oomd[795]: No swap; memory pressure usage will be degraded868vm-test-run-nixbot-gitlab> gitlab # [    7.089497] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore869vm-test-run-nixbot-gitlab> gitlab # [    7.092729] systemd[1]: Starting Load/Save OS Random Seed...870vm-test-run-nixbot-gitlab> gitlab # [    7.095414] systemd[1]: Starting Apply Kernel Variables...871vm-test-run-nixbot-gitlab> gitlab # [    7.099531] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...872vm-test-run-nixbot-gitlab> gitlab # [    7.101792] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki873vm-test-run-nixbot-gitlab> gitlab # [    7.367226] systemd-journald[792]: Received client request to flush runtime journal.874vm-test-run-nixbot-gitlab> gitlab # [    7.167174] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.875vm-test-run-nixbot-gitlab> gitlab # [    7.171231] systemd[1]: Mounted Kernel Configuration File System.876vm-test-run-nixbot-gitlab> gitlab # [    7.173489] systemd[1]: Finished Load/Save OS Random Seed.877vm-test-run-nixbot-gitlab> gitlab # [    7.175755] systemd[1]: Reached target First Boot Complete.878vm-test-run-nixbot-gitlab> gitlab # [    7.177441] systemd[1]: Finished Apply Kernel Variables.879vm-test-run-nixbot-gitlab> gitlab # [    7.178760] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.880vm-test-run-nixbot-gitlab> gitlab # [    7.180667] systemd[1]: Starting Create Static Device Nodes in /dev...881vm-test-run-nixbot-gitlab> gitlab # [    7.182256] systemd[1]: Finished Create Static Device Nodes in /dev.882vm-test-run-nixbot-gitlab> gitlab # [    7.183819] systemd[1]: Reached target Preparation for Local File Systems.883vm-test-run-nixbot-gitlab> gitlab # [    7.185721] systemd[1]: Starting Rule-based Manager for Device Events and Files...884vm-test-run-nixbot-gitlab> gitlab # [    7.187543] systemd[1]: Finished Flush Journal to Persistent Storage.885vm-test-run-nixbot-gitlab> gitlab # [    7.190627] systemd-udevd[827]: Using default interface naming scheme 'v260'.886vm-test-run-nixbot-gitlab> gitlab # [    7.231124] systemd[1]: Started Rule-based Manager for Device Events and Files.887vm-test-run-nixbot-gitlab> gitlab # [    7.259088] systemd[1]: Finished Coldplug All udev Devices.888vm-test-run-nixbot-gitlab> gitlab # [    7.355636] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse889vm-test-run-nixbot-gitlab> gitlab # [    7.383472] systemd[1]: Finished Firewall.890vm-test-run-nixbot-gitlab> gitlab # [    7.395050] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.891vm-test-run-nixbot-gitlab> gitlab # [    7.403799] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped.892vm-test-run-nixbot-gitlab> gitlab # [    7.405816] (udev-worker)[909]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.893vm-test-run-nixbot-gitlab> gitlab # [    7.409522] (udev-worker)[909]: Network interface NamePolicy= disabled on kernel command line.894vm-test-run-nixbot-gitlab> gitlab # [    7.411772] (udev-worker)[922]: Network interface NamePolicy= disabled on kernel command line.895vm-test-run-nixbot-gitlab> gitlab # [    7.428485] systemd[1]: Condition check resulted in Virtio network device being skipped.896vm-test-run-nixbot-gitlab> gitlab # [    7.430613] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore897vm-test-run-nixbot-gitlab> gitlab # [    7.433336] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67898vm-test-run-nixbot-gitlab> gitlab # [    7.437182] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore899vm-test-run-nixbot-gitlab> gitlab # [    7.440285] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki900vm-test-run-nixbot-gitlab> gitlab # [    7.442616] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki901vm-test-run-nixbot-gitlab> gitlab # [    7.726590] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3902vm-test-run-nixbot-gitlab> gitlab # [    7.733038] mousedev: PS/2 mouse device common for all mice903vm-test-run-nixbot-gitlab> gitlab # [    7.739855] ACPI: button: Power Button [PWRF]904vm-test-run-nixbot-gitlab> gitlab # [    7.770279] rtc_cmos 00:05: RTC can wake from S4905vm-test-run-nixbot-gitlab> gitlab # [    7.773770] Floppy drive(s): fd0 is 2.88M AMI BIOS906vm-test-run-nixbot-gitlab> gitlab # [    7.786492] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0907vm-test-run-nixbot-gitlab> gitlab # [    7.786969] FDC 0 is a S82078B908vm-test-run-nixbot-gitlab> gitlab # [    7.788082] rtc_cmos 00:05: registered as rtc0909vm-test-run-nixbot-gitlab> gitlab # [    7.788389] rtc_cmos 00:05: setting system clock to 2026-06-29T10:43:15 UTC (1782729795)910vm-test-run-nixbot-gitlab> gitlab # [    7.788526] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD911vm-test-run-nixbot-gitlab> gitlab # [    7.788535] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs912vm-test-run-nixbot-gitlab> gitlab # [    7.798043] parport_pc 00:03: reported by Plug and Play ACPI913vm-test-run-nixbot-gitlab> gitlab # [    7.799934] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)]914vm-test-run-nixbot-gitlab> gitlab # [    7.805802] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input4915vm-test-run-nixbot-gitlab> gitlab # [    7.812455] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console916vm-test-run-nixbot-gitlab> gitlab # [    7.813362] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6917vm-test-run-nixbot-gitlab> gitlab # [    7.831323] Console: switching to colour dummy device 80x25918vm-test-run-nixbot-gitlab> gitlab # [    7.832667] [drm] Found bochs VGA, ID 0xb0c5.919vm-test-run-nixbot-gitlab> gitlab # [    7.833534] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000.920vm-test-run-nixbot-gitlab> gitlab # [    7.838669] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5921vm-test-run-nixbot-gitlab> gitlab # [    7.850492] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic922vm-test-run-nixbot-gitlab> gitlab # [    7.851831] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0923vm-test-run-nixbot-gitlab> gitlab # [    7.879320] fbcon: bochs-drmdrmfb (fb0) is primary device924vm-test-run-nixbot-gitlab> gitlab # [    7.896455] Console: switching to colour frame buffer device 160x50925vm-test-run-nixbot-gitlab> gitlab # [    7.637685] systemd[1]: Starting Virtual Console Setup...926vm-test-run-nixbot-gitlab> gitlab # [    7.913021] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device927vm-test-run-nixbot-gitlab> gitlab # [    7.681486] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.928vm-test-run-nixbot-gitlab> gitlab # [    7.683447] systemd[1]: Stopped Virtual Console Setup.929vm-test-run-nixbot-gitlab> gitlab # [    7.692118] systemd[1]: Starting Virtual Console Setup...930vm-test-run-nixbot-gitlab> gitlab # [    7.701565] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.931vm-test-run-nixbot-gitlab> gitlab # [    7.704133] systemd[1]: Stopped Virtual Console Setup.932vm-test-run-nixbot-gitlab> gitlab # [    7.710152] systemd[1]: Starting Virtual Console Setup...933vm-test-run-nixbot-gitlab> gitlab # [    7.988202] ppdev: user-space parallel port driver934vm-test-run-nixbot-gitlab> gitlab # [    8.013371] kvm_amd: TSC scaling supported935vm-test-run-nixbot-gitlab> gitlab # [    8.014463] kvm_amd: Nested Virtualization enabled936vm-test-run-nixbot-gitlab> gitlab # [    8.016006] kvm_amd: Nested Paging enabled937vm-test-run-nixbot-gitlab> gitlab # [    8.017477] kvm_amd: LBR virtualization supported938vm-test-run-nixbot-gitlab> gitlab # [    8.018792] kvm_amd: Virtual VMLOAD VMSAVE supported939vm-test-run-nixbot-gitlab> gitlab # [    8.020032] kvm_amd: Virtual GIF supported940vm-test-run-nixbot-gitlab> gitlab # [    8.021077] kvm_amd: Virtual NMI enabled941vm-test-run-nixbot-gitlab> gitlab # [    8.072280] EDAC MC: Ver: 3.0.0942vm-test-run-nixbot-gitlab> gitlab # [    7.810919] systemd-vconsole-setup[965]: Configuration of first virtual console was skipped, ignoring remaining ones.943vm-test-run-nixbot-gitlab> gitlab # [    7.813966] systemd[1]: Finished Virtual Console Setup.944vm-test-run-nixbot-gitlab> gitlab # [    7.870598] systemd[1]: Mounting /run/wrappers...945vm-test-run-nixbot-gitlab> gitlab # [    7.888716] systemd[1]: Mounted /run/wrappers.946vm-test-run-nixbot-gitlab> gitlab # [    7.891127] systemd[1]: Reached target Local File Systems.947vm-test-run-nixbot-gitlab> gitlab # [    7.893843] systemd[1]: Listening on Boot Loader Control Service Socket.948vm-test-run-nixbot-gitlab> gitlab # [    7.896937] systemd[1]: Starting register-nix-paths.service...949vm-test-run-nixbot-gitlab> gitlab # [    7.899732] systemd[1]: Starting Create SUID/SGID Wrappers...950vm-test-run-nixbot-gitlab> gitlab # [    7.902517] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.951vm-test-run-nixbot-gitlab> gitlab # [    7.906648] systemd[1]: Starting Save Transient machine-id to Disk...952vm-test-run-nixbot-gitlab> gitlab # [    7.909465] systemd[1]: Starting Create System Files and Directories...953vm-test-run-nixbot-gitlab> gitlab # [    7.941303] systemd[1]: Finished Create System Files and Directories.954vm-test-run-nixbot-gitlab> gitlab # [    7.944561] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.955vm-test-run-nixbot-gitlab> gitlab # [    7.949299] systemd[1]: Finished Save Transient machine-id to Disk.956vm-test-run-nixbot-gitlab> gitlab # [    7.951293] systemd[1]: Starting Rebuild Journal Catalog...957vm-test-run-nixbot-gitlab> gitlab # [    7.954655] systemd[1]: Starting Record System Boot/Shutdown in UTMP...958vm-test-run-nixbot-gitlab> gitlab # [    7.972170] systemd[1]: Finished Record System Boot/Shutdown in UTMP.959vm-test-run-nixbot-gitlab> gitlab # [    7.988041] systemd[1]: Finished Rebuild Journal Catalog.960vm-test-run-nixbot-gitlab> gitlab # [    7.989918] systemd[1]: Starting Update is Completed...961vm-test-run-nixbot-gitlab> gitlab # [    8.006129] systemd[1]: Finished Update is Completed.962vm-test-run-nixbot-gitlab> gitlab # [    8.045566] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.963vm-test-run-nixbot-gitlab> gitlab # [    8.047384] systemd[1]: Finished Create SUID/SGID Wrappers.964vm-test-run-nixbot-gitlab> gitlab # [    8.048746] systemd[1]: suid-sgid-wrappers.service: Consumed 115ms CPU time over 149ms wall clock time, 4.5M memory peak, 1.1M read from disk.965vm-test-run-nixbot-gitlab> gitlab # [    8.309529] systemd[1]: Finished register-nix-paths.service.966vm-test-run-nixbot-gitlab> gitlab # [    8.312389] systemd[1]: Reached target System Initialization.967vm-test-run-nixbot-gitlab> gitlab # [    8.315180] systemd[1]: Started Discard unused filesystem blocks once a week.968vm-test-run-nixbot-gitlab> gitlab # [    8.318647] systemd[1]: Started logrotate.timer.969vm-test-run-nixbot-gitlab> gitlab # [    8.320852] systemd[1]: Started Daily Cleanup of Temporary Directories.970vm-test-run-nixbot-gitlab> gitlab # [    8.324004] systemd[1]: Reached target Timer Units.971vm-test-run-nixbot-gitlab> gitlab # [    8.326308] systemd[1]: Listening on D-Bus System Message Bus Socket.972vm-test-run-nixbot-gitlab> gitlab # [    8.329334] systemd[1]: Listening on Nix Daemon Socket.973vm-test-run-nixbot-gitlab> gitlab # [    8.331803] systemd[1]: Starting nixbot CI service socket...974vm-test-run-nixbot-gitlab> gitlab # [    8.334563] systemd[1]: Listening on Hostname Service Socket.975vm-test-run-nixbot-gitlab> gitlab # [    8.337317] systemd[1]: Listening on nixbot CI service socket.976vm-test-run-nixbot-gitlab> gitlab # [    8.340109] systemd[1]: Reached target Socket Units.977vm-test-run-nixbot-gitlab> gitlab # [    8.342467] systemd[1]: Starting D-Bus System Message Bus...978vm-test-run-nixbot-gitlab> gitlab # [    8.399743] dbus-broker-launch[1055]: Looking up NSS user entry for 'systemd-timesync'...979vm-test-run-nixbot-gitlab> gitlab # [    8.404414] dbus-broker-launch[1055]: NSS returned no entry for 'systemd-timesync'980vm-test-run-nixbot-gitlab> gitlab # [    8.408089] dbus-broker-launch[1055]: Invalid user-name in /nix/store/89zj9nmrajvcy9c3c08wv0ir9ww7lgmq-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"981vm-test-run-nixbot-gitlab> gitlab # [    8.416629] systemd[1]: Started D-Bus System Message Bus.982vm-test-run-nixbot-gitlab> gitlab # [    8.437148] dbus-broker-launch[1055]: Ready983vm-test-run-nixbot-gitlab> gitlab # [    8.441553] systemd[1]: Reached target Basic System.984vm-test-run-nixbot-gitlab> gitlab # [    8.444496] systemd[1]: Started backdoor.service.985vm-test-run-nixbot-gitlab> gitlab # [    8.449314] systemd[1]: Starting gitlab-config.service...986vm-test-run-nixbot-gitlab> gitlab # [    8.452529] systemd[1]: Starting Import lastlog data into lastlog2 database...987vm-test-run-nixbot-gitlab> gitlab # [    8.461459] systemd[1]: Starting Logrotate configuration check...988vm-test-run-nixbot-gitlab> gitlab # [    8.470231] systemd[1]: Starting Name Service Cache Daemon (nsncd)...989vm-test-run-nixbot-gitlab> gitlab # [    8.475903] systemd[1]: Starting Post-Boot Actions...990vm-test-run-nixbot-gitlab> gitlab # connecting to host...991vm-test-run-nixbot-gitlab> gitlab # [    8.480400] systemd[1]: Started Reset console on configuration changes.992vm-test-run-nixbot-gitlab> gitlab # [    8.483835] logrotate[1060]: warning: logrotate in debug mode does nothing except printing debug messages!  Consider using verbose mode (-v) instead if this is not what you want.993vm-test-run-nixbot-gitlab> gitlab # [    8.486677] logrotate[1060]: reading config file /nix/store/sn4bd2nqi3yg4r51djm9i2rmfgbwvmhb-logrotate.conf994vm-test-run-nixbot-gitlab> gitlab # [    8.488786] logrotate[1060]: note: 'monthly' overrides previously specified 'weekly'995vm-test-run-nixbot-gitlab> gitlab # [    8.490850] logrotate[1060]: Reading state from file: /var/lib/logrotate.status996vm-test-run-nixbot-gitlab> gitlab # [    8.493655] logrotate[1060]: state file /var/lib/logrotate.status does not exist997vm-test-run-nixbot-gitlab> gitlab # [    8.495189] logrotate[1060]: Allocating hash table for state file, size 64 entries998vm-test-run-nixbot-gitlab> gitlab # [    8.497403] logrotate[1060]: Handling 4 logs999vm-test-run-nixbot-gitlab> gitlab # [    8.498231] logrotate[1060]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed1000vm-test-run-nixbot-gitlab> gitlab # [    8.499129] logrotate[1060]: considering log /var/log/btmp1001vm-test-run-nixbot-gitlab> gitlab # [    8.499925] logrotate[1060]: Creating new state1002vm-test-run-nixbot-gitlab> gitlab # [    8.501115] logrotate[1060]:   Now: 2026-06-29 10:431003vm-test-run-nixbot-gitlab> gitlab # [    8.501956] logrotate[1060]:   Last rotated at 2026-06-29 10:001004vm-test-run-nixbot-gitlab> gitlab # [    8.503130] logrotate[1060]:   log does not need rotating (log has already been rotated)1005vm-test-run-nixbot-gitlab> gitlab # [    8.503700] logrotate[1060]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed1006vm-test-run-nixbot-gitlab> gitlab # [    8.504912] logrotate[1060]: considering log /var/log/wtmp1007vm-test-run-nixbot-gitlab> gitlab # [    8.505731] logrotate[1060]: Creating new state1008vm-test-run-nixbot-gitlab> gitlab # [    8.506881] logrotate[1060]:   Now: 2026-06-29 10:431009vm-test-run-nixbot-gitlab> gitlab # [    8.508055] logrotate[1060]:   Last rotated at 2026-06-29 10:001010vm-test-run-nixbot-gitlab> gitlab # [    8.508873] logrotate[1060]:   log does not need rotating (log has already been rotated)1011vm-test-run-nixbot-gitlab> gitlab # [    8.510060] logrotate[1060]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed1012vm-test-run-nixbot-gitlab> gitlab # [    8.510895] logrotate[1060]: switching euid from 0 to 165 and egid from 0 to 165 (pid 1060)1013vm-test-run-nixbot-gitlab> gitlab # [    8.512077] logrotate[1060]: considering log /var/gitlab/state/log/*.log1014vm-test-run-nixbot-gitlab> gitlab # [    8.512928] logrotate[1060]:   log /var/gitlab/state/log/*.log does not exist -- skipping1015vm-test-run-nixbot-gitlab> gitlab # [    8.514101] logrotate[1060]: Creating new state1016vm-test-run-nixbot-gitlab> gitlab # [    8.514649] logrotate[1060]: switching euid from 165 to 0 and egid from 165 to 0 (pid 1060)1017vm-test-run-nixbot-gitlab> gitlab # [    8.515812] logrotate[1060]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed1018vm-test-run-nixbot-gitlab> gitlab # [    8.516664] logrotate[1060]: switching euid from 0 to 60 and egid from 0 to 60 (pid 1060)1019vm-test-run-nixbot-gitlab> gitlab # [    8.517830] logrotate[1060]: considering log /var/log/nginx/*.log1020vm-test-run-nixbot-gitlab> gitlab # [    8.518972] logrotate[1060]:   log /var/log/nginx/*.log does not exist -- skipping1021vm-test-run-nixbot-gitlab> gitlab # [    8.520139] logrotate[1060]: Creating new state1022vm-test-run-nixbot-gitlab> gitlab # [    8.520960] logrotate[1060]: switching euid from 60 to 0 and egid from 60 to 0 (pid 1060)1023vm-test-run-nixbot-gitlab> gitlab # [    8.530293] systemd[1]: Starting resolvconf update...1024vm-test-run-nixbot-gitlab> gitlab: Guest shell says: b'Spawning backdoor root shell...\n'1025vm-test-run-nixbot-gitlab> gitlab: connected to guest root shell1026vm-test-run-nixbot-gitlab> gitlab: (connecting took 9.40 seconds)1027vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for the VM to finish booting, in 9.40 seconds)1028vm-test-run-nixbot-gitlab> gitlab # [    8.553832] systemd[1]: Finished Logrotate configuration check.1029vm-test-run-nixbot-gitlab> gitlab # [    8.555115] systemd[1]: Finished Post-Boot Actions.1030vm-test-run-nixbot-gitlab> gitlab # [    8.556877] systemd[1]: Finished Import lastlog data into lastlog2 database.1031vm-test-run-nixbot-gitlab> gitlab # [    8.566940] nsncd[1062]: Jun 29 10:43:16.540 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"1032vm-test-run-nixbot-gitlab> gitlab # [    8.570822] systemd[1]: Started Name Service Cache Daemon (nsncd).1033vm-test-run-nixbot-gitlab> gitlab # [    8.572533] systemd[1]: Reached target Host and Network Name Lookups.1034vm-test-run-nixbot-gitlab> gitlab # [    8.574224] systemd[1]: Reached target User and Group Name Lookups.1035vm-test-run-nixbot-gitlab> gitlab # [    8.575871] systemd[1]: Starting User Login Management...1036vm-test-run-nixbot-gitlab> gitlab # [    8.606329] systemd[1]: Stopped target Host and Network Name Lookups.1037vm-test-run-nixbot-gitlab> gitlab # [    8.608110] systemd[1]: Stopping Host and Network Name Lookups...1038vm-test-run-nixbot-gitlab> gitlab # [    8.609829] systemd[1]: Stopped target User and Group Name Lookups.1039vm-test-run-nixbot-gitlab> gitlab # [    8.611749] systemd[1]: Stopping User and Group Name Lookups...1040vm-test-run-nixbot-gitlab> gitlab # [    8.613392] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...1041vm-test-run-nixbot-gitlab> gitlab # [    8.615108] systemd[1]: nscd.service: Deactivated successfully.1042vm-test-run-nixbot-gitlab> gitlab # [    8.616766] systemd[1]: Stopped Name Service Cache Daemon (nsncd).1043vm-test-run-nixbot-gitlab> gitlab # [    8.618511] systemd[1]: nscd.service: Consumed 74ms CPU time over 140ms wall clock time, 8M memory peak, 5.7M read from disk.1044vm-test-run-nixbot-gitlab> gitlab # [    8.622551] systemd[1]: Starting Name Service Cache Daemon (nsncd)...1045vm-test-run-nixbot-gitlab> gitlab # [    8.624640] systemd-logind[1109]: New seat seat0.1046vm-test-run-nixbot-gitlab> gitlab # [    8.626790] systemd-logind[1109]: Watching system buttons on /dev/input/event2 (Power Button)1047vm-test-run-nixbot-gitlab> gitlab # [    8.629123] systemd-logind[1109]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard)1048vm-test-run-nixbot-gitlab> gitlab # [    8.631827] systemd-logind[1109]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard)1049vm-test-run-nixbot-gitlab> gitlab # [    8.634501] systemd[1]: Started User Login Management.1050vm-test-run-nixbot-gitlab> gitlab # [    8.635906] systemd[1]: Starting linger-users.service...1051vm-test-run-nixbot-gitlab> gitlab # [    8.643070] systemd[1]: Started Name Service Cache Daemon (nsncd).1052vm-test-run-nixbot-gitlab> gitlab # [    8.644650] nsncd[1145]: Jun 29 10:43:16.616 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"1053vm-test-run-nixbot-gitlab> gitlab # [    8.646975] systemd[1]: Reached target Host and Network Name Lookups.1054vm-test-run-nixbot-gitlab> gitlab # [    8.648119] systemd[1]: Reached target User and Group Name Lookups.1055vm-test-run-nixbot-gitlab> gitlab # [    8.651411] systemd[1]: linger-users.service: Deactivated successfully.1056vm-test-run-nixbot-gitlab> gitlab # [    8.652754] systemd[1]: Finished linger-users.service.1057vm-test-run-nixbot-gitlab> gitlab # [    8.658714] systemd[1]: Finished resolvconf update.1058vm-test-run-nixbot-gitlab> gitlab # [    8.660092] systemd[1]: Reached target Preparation for Network.1059vm-test-run-nixbot-gitlab> gitlab # [    8.662825] systemd[1]: Starting DHCP Client...1060vm-test-run-nixbot-gitlab> gitlab # [    8.664724] systemd[1]: Starting Address configuration of eth1...1061vm-test-run-nixbot-gitlab> gitlab # [    8.669504] systemd[1]: Starting Extra networking commands....1062vm-test-run-nixbot-gitlab> gitlab # [    8.692763] network-addresses-eth1-start[1173]: adding address 192.168.1.1/24... done1063vm-test-run-nixbot-gitlab> gitlab # [    8.697733] network-addresses-eth1-start[1173]: adding address 2001:db8:1::1/64... done1064vm-test-run-nixbot-gitlab> gitlab # [    8.708475] systemd[1]: Finished Address configuration of eth1.1065vm-test-run-nixbot-gitlab> gitlab # [    8.733052] systemd[1]: Finished Extra networking commands..1066vm-test-run-nixbot-gitlab> gitlab # [    8.735816] systemd[1]: Reached target Network.1067vm-test-run-nixbot-gitlab> gitlab # [    8.738514] systemd[1]: Starting gitlab-workhorse.service...1068vm-test-run-nixbot-gitlab> gitlab # [    8.741698] dhcpcd[1188]: dhcpcd-10.3.2 starting1069vm-test-run-nixbot-gitlab> gitlab # [    8.743333] systemd[1]: Starting Nginx Web Server...1070vm-test-run-nixbot-gitlab> gitlab # [    8.745975] systemd[1]: Starting PostgreSQL Server...1071vm-test-run-nixbot-gitlab> gitlab # [    8.747417] dhcpcd[1233]: dev: loaded udev1072vm-test-run-nixbot-gitlab> gitlab # [    8.750984] systemd[1]: Starting Redis Server - redis-gitlab...1073vm-test-run-nixbot-gitlab> gitlab # [    8.752441] systemd[1]: Starting Permit User Sessions...1074vm-test-run-nixbot-gitlab> gitlab # [    9.018175] 8021q: 802.1Q VLAN Support v1.81075vm-test-run-nixbot-gitlab> gitlab # [    9.019141] 8021q: adding VLAN 0 to HW filter on device eth11076vm-test-run-nixbot-gitlab> gitlab # [    8.778245] systemd[1]: Finished Permit User Sessions.1077vm-test-run-nixbot-gitlab> gitlab # [    8.784917] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.1078vm-test-run-nixbot-gitlab> gitlab # [    8.791094] systemd[1]: Started Getty on tty1.1079vm-test-run-nixbot-gitlab> gitlab # [    8.792254] systemd[1]: Reached target Login Prompts.1080vm-test-run-nixbot-gitlab> gitlab # [    9.122479] cfg80211: Loading compiled-in X.509 certificates for regulatory database1081vm-test-run-nixbot-gitlab> gitlab # [    9.127641] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'1082vm-test-run-nixbot-gitlab> gitlab # [    9.129267] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'1083vm-test-run-nixbot-gitlab> gitlab # [    9.131999] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -21084vm-test-run-nixbot-gitlab> gitlab # [    9.134438] cfg80211: failed to load regulatory.db1085vm-test-run-nixbot-gitlab> gitlab # [    8.881070] nginx-pre-start[1252]: nginx: the configuration file /nix/store/nlpb2pnbsk0921apkv5yml2qxbm30d7w-nginx.conf syntax is ok1086vm-test-run-nixbot-gitlab> gitlab # [    8.882366] nginx-pre-start[1252]: nginx: configuration file /nix/store/nlpb2pnbsk0921apkv5yml2qxbm30d7w-nginx.conf test is successful1087vm-test-run-nixbot-gitlab> gitlab # [    8.890399] systemd[1]: Started Nginx Web Server.1088vm-test-run-nixbot-gitlab> gitlab # [    9.166041] 8021q: adding VLAN 0 to HW filter on device eth01089vm-test-run-nixbot-gitlab> gitlab # [    8.906119] dhcpcd[1233]: eth0: waiting for carrier1090vm-test-run-nixbot-gitlab> gitlab # [    8.907611] dhcpcd[1233]: eth0: carrier acquired1091vm-test-run-nixbot-gitlab> gitlab # [    8.911386] dhcpcd[1233]: DUID 00:01:00:01:31:d5:08:c4:52:54:00:12:34:561092vm-test-run-nixbot-gitlab> gitlab # [    8.912553] dhcpcd[1233]: eth0: IAID 00:12:34:561093vm-test-run-nixbot-gitlab> gitlab # [    8.913548] dhcpcd[1233]: eth0: adding address fe80::5054:ff:fe12:34561094vm-test-run-nixbot-gitlab> gitlab # [    8.922905] redis-server[1256]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.1095vm-test-run-nixbot-gitlab> gitlab # [    8.926553] redis-server[1256]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo1096vm-test-run-nixbot-gitlab> gitlab # [    8.928255] redis-server[1256]: Redis version=8.8.0, bits=64, commit=00000000, modified=0, pid=1256, just started1097vm-test-run-nixbot-gitlab> gitlab # [    8.931591] redis-server[1256]: Configuration loaded1098vm-test-run-nixbot-gitlab> gitlab # [    8.932978] redis[1256]: monotonic clock: POSIX clock_gettime1099vm-test-run-nixbot-gitlab> gitlab # [    8.934147] systemd[1]: Started Redis Server - redis-gitlab.1100vm-test-run-nixbot-gitlab> gitlab # [    8.935143] redis[1256]: Running mode=standalone, port=0.1101vm-test-run-nixbot-gitlab> gitlab # [    8.937686] redis[1256]: Server initialized1102vm-test-run-nixbot-gitlab> gitlab # [    8.938742] redis[1256]: Ready to accept connections unix1103vm-test-run-nixbot-gitlab> gitlab # [    8.939953] redis[1256]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.1104vm-test-run-nixbot-gitlab> gitlab # [    8.946763] postgresql-pre-start[1258]: The files belonging to this database system will be owned by user "postgres".1105vm-test-run-nixbot-gitlab> gitlab # [    8.950609] postgresql-pre-start[1258]: This user must also own the server process.1106vm-test-run-nixbot-gitlab> gitlab # [    8.953310] postgresql-pre-start[1258]: The database cluster will be initialized with locale "en_US.UTF-8".1107vm-test-run-nixbot-gitlab> gitlab # [    8.954359] postgresql-pre-start[1258]: The default database encoding has accordingly been set to "UTF8".1108vm-test-run-nixbot-gitlab> gitlab # [    8.954935] postgresql-pre-start[1258]: The default text search configuration will be set to "english".1109vm-test-run-nixbot-gitlab> gitlab # [    8.957095] postgresql-pre-start[1258]: Data page checksums are disabled.1110vm-test-run-nixbot-gitlab> gitlab # [    8.958686] postgresql-pre-start[1258]: fixing permissions on existing directory /var/lib/postgresql/17 ... ok1111vm-test-run-nixbot-gitlab> gitlab # [    8.959807] postgresql-pre-start[1258]: creating subdirectories ... ok1112vm-test-run-nixbot-gitlab> gitlab # [    8.961157] postgresql-pre-start[1258]: selecting dynamic shared memory implementation ... posix1113vm-test-run-nixbot-gitlab> gitlab # [    9.000367] postgresql-pre-start[1258]: selecting default "max_connections" ... 1001114vm-test-run-nixbot-gitlab> gitlab # [    9.044694] postgresql-pre-start[1258]: selecting default "shared_buffers" ... 128MB1115vm-test-run-nixbot-gitlab> gitlab # [    9.225946] systemd[1]: Started gitlab-workhorse.service.1116vm-test-run-nixbot-gitlab> gitlab # [    9.445937] postgresql-pre-start[1258]: selecting default time zone ... UTC1117vm-test-run-nixbot-gitlab> gitlab # [    9.448549] postgresql-pre-start[1258]: creating configuration files ... ok1118vm-test-run-nixbot-gitlab> gitlab # [    9.511671] gitlab-workhorse[1291]: time="2026-06-29T10:43:17Z" level=info msg=Starting build_time=19700101.000000 version=18.11.41119vm-test-run-nixbot-gitlab> gitlab # [    9.513510] gitlab-workhorse[1291]: time="2026-06-29T10:43:17Z" level=info msg="Using redis/go-redis"1120vm-test-run-nixbot-gitlab> gitlab # [    9.515232] gitlab-workhorse[1291]: time="2026-06-29T10:43:17Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix1121vm-test-run-nixbot-gitlab> gitlab # [    9.590536] dhcpcd[1233]: eth0: soliciting a DHCP lease1122vm-test-run-nixbot-gitlab> gitlab # [    9.859757] NET: Registered PF_PACKET protocol family1123vm-test-run-nixbot-gitlab> gitlab # [    9.607530] dhcpcd[1233]: eth0: offered 10.0.2.15 from 10.0.2.21124vm-test-run-nixbot-gitlab> gitlab # [    9.612308] dhcpcd[1233]: eth0: probing address 10.0.2.15/241125vm-test-run-nixbot-gitlab> gitlab # [    9.621119] postgresql-pre-start[1258]: running bootstrap script ... ok1126vm-test-run-nixbot-gitlab> gitlab # [   10.105459] postgresql-pre-start[1258]: performing post-bootstrap initialization ... ok1127vm-test-run-nixbot-gitlab> gitlab # [   10.476758] postgresql-pre-start[1258]: syncing data to disk ... ok1128vm-test-run-nixbot-gitlab> gitlab # [   10.480258] postgresql-pre-start[1258]: initdb: warning: enabling "trust" authentication for local connections1129vm-test-run-nixbot-gitlab> gitlab # [   10.482854] postgresql-pre-start[1258]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.1130vm-test-run-nixbot-gitlab> gitlab # [   10.484995] postgresql-pre-start[1258]: Success. You can now start the database server using:1131vm-test-run-nixbot-gitlab> gitlab # [   10.488139] postgresql-pre-start[1258]:     pg_ctl -D /var/lib/postgresql/17 -l logfile start1132vm-test-run-nixbot-gitlab> gitlab # [   10.609513] postgres[1318]: [1318] LOG:  starting PostgreSQL 17.10 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit1133vm-test-run-nixbot-gitlab> gitlab # [   10.611459] postgres[1318]: [1318] LOG:  listening on IPv6 address "::1", port 54321134vm-test-run-nixbot-gitlab> gitlab # [   10.613142] postgres[1318]: [1318] LOG:  listening on IPv4 address "127.0.0.1", port 54321135vm-test-run-nixbot-gitlab> gitlab # [   10.614971] postgres[1318]: [1318] LOG:  listening on Unix socket "/run/postgresql/.s.PGSQL.5432"1136vm-test-run-nixbot-gitlab> gitlab # [   10.625442] postgres[1324]: [1324] LOG:  database system was shut down at 2026-06-29 10:43:18 GMT1137vm-test-run-nixbot-gitlab> gitlab # [   10.637444] postgres[1318]: [1318] LOG:  database system is ready to accept connections1138vm-test-run-nixbot-gitlab> gitlab # [   10.646399] systemd[1]: Started PostgreSQL Server.1139vm-test-run-nixbot-gitlab> gitlab # [   10.650973] systemd[1]: Starting PostgreSQL Setup Scripts...1140vm-test-run-nixbot-gitlab> gitlab # [   10.789423] postgresql-setup-start[1344]: CREATE DATABASE1141vm-test-run-nixbot-gitlab> gitlab # [   10.827709] postgresql-setup-start[1349]: CREATE ROLE1142vm-test-run-nixbot-gitlab> gitlab # [   10.842868] postgresql-setup-start[1351]: ALTER DATABASE1143vm-test-run-nixbot-gitlab> gitlab # [   10.873427] postgresql-setup-start[1360]: CREATE ROLE1144vm-test-run-nixbot-gitlab> gitlab # [   10.877641] systemd[1]: Finished PostgreSQL Setup Scripts.1145vm-test-run-nixbot-gitlab> gitlab # [   10.880141] systemd[1]: Reached target PostgreSQL.1146vm-test-run-nixbot-gitlab> gitlab # [   10.882910] systemd[1]: Starting gitlab-postgresql.service...1147vm-test-run-nixbot-gitlab> gitlab # [   10.988443] 119c5p75zqwmpdrsjsd5cszxwrq2hgfq-gitlab-config[1356]: mkdir -p /var/gitlab/state/home/.ssh: OK1148vm-test-run-nixbot-gitlab> gitlab # [   10.995524] 119c5p75zqwmpdrsjsd5cszxwrq2hgfq-gitlab-config[1356]: chmod 700 /var/gitlab/state/home/.ssh: OK1149vm-test-run-nixbot-gitlab> gitlab # [   11.001749] gitlab-postgresql-start[1369]: CREATE DATABASE1150vm-test-run-nixbot-gitlab> gitlab # [   11.028933] gitlab-postgresql-start[1379]: CREATE EXTENSION1151vm-test-run-nixbot-gitlab> gitlab # [   11.037628] systemd[1]: Finished gitlab-config.service.1152vm-test-run-nixbot-gitlab> gitlab # [   11.044287] systemd[1]: Started gitaly.service.1153vm-test-run-nixbot-gitlab> gitlab # [   11.047736] systemd[1]: Starting gitlab-db-config.service...1154vm-test-run-nixbot-gitlab> gitlab # [   11.053712] dhcpcd[1233]: eth0: soliciting an IPv6 router1155vm-test-run-nixbot-gitlab> gitlab # [   11.055454] dhcpcd[1233]: eth0: Router Advertisement from fe80::21156vm-test-run-nixbot-gitlab> gitlab # [   11.057576] dhcpcd[1233]: eth0: adding address fec0::5054:ff:fe12:3456/641157vm-test-run-nixbot-gitlab> gitlab # [   11.061384] dhcpcd[1233]: eth0: adding route to fec0::/641158vm-test-run-nixbot-gitlab> gitlab # [   11.063658] dhcpcd[1233]: eth0: adding default route via fe80::21159vm-test-run-nixbot-gitlab> gitlab # [   11.065783] gitlab-postgresql-start[1386]: CREATE EXTENSION1160vm-test-run-nixbot-gitlab> gitlab # [   11.078861] systemd[1]: Finished gitlab-postgresql.service.1161vm-test-run-nixbot-gitlab> gitlab # [   11.611554] gitaly[1389]: time="2026-06-29T10:43:19.584Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/2wcq0nlg508fs5wq7jnpg1lw2pib950g-gitaly.toml' and fix the errors that are printed." pid=13891162vm-test-run-nixbot-gitlab> gitlab # [   11.628247] gitaly[1389]: time="2026-06-29T10:43:19.584Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=13891163vm-test-run-nixbot-gitlab> gitlab # [   11.634803] gitaly[1389]: time="2026-06-29T10:43:19.584Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=13891164vm-test-run-nixbot-gitlab> gitlab # [   11.642625] gitaly[1389]: time="2026-06-29T10:43:19.584Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=13891165vm-test-run-nixbot-gitlab> gitlab # [   11.648878] gitaly[1389]: time="2026-06-29T10:43:19.584Z" level=info msg="Starting Gitaly" pid=1389 version=18.11.41166vm-test-run-nixbot-gitlab> gitlab # [   11.783126] gitaly[1389]: time="2026-06-29T10:43:19.756Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=171 pid=13891167vm-test-run-nixbot-gitlab> gitlab # [   11.784393] gitaly[1389]: time="2026-06-29T10:43:19.756Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=13891168vm-test-run-nixbot-gitlab> gitlab # [   11.785788] gitaly[1389]: time="2026-06-29T10:43:19.759Z" level=info msg="finished initializing command factory" duration_ms=2 pid=13891169vm-test-run-nixbot-gitlab> gitlab # [   11.786877] gitaly[1389]: time="2026-06-29T10:43:19.759Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-1389/git-exec-189983346.d/git pid=13891170vm-test-run-nixbot-gitlab> gitlab # [   11.797718] gitaly[1389]: time="2026-06-29T10:43:19.771Z" level=info msg="finished detecting git version" duration_ms=12 pid=13891171vm-test-run-nixbot-gitlab> gitlab # [   11.798572] gitaly[1389]: time="2026-06-29T10:43:19.771Z" level=info msg="using Git version" pid=1389 version=2.53.g1172vm-test-run-nixbot-gitlab> gitlab # [   11.804244] gitaly[1389]: time="2026-06-29T10:43:19.777Z" level=info msg="clearing disk cache object folder" pid=1389 storage=default1173vm-test-run-nixbot-gitlab> gitlab # [   11.807751] gitaly[1389]: time="2026-06-29T10:43:19.778Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache235722046 pid=1389 storage=default1174vm-test-run-nixbot-gitlab> gitlab # [   11.808803] gitaly[1389]: time="2026-06-29T10:43:19.778Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache235722046 pid=1389 storage=default1175vm-test-run-nixbot-gitlab> gitlab # [   11.809764] gitaly[1389]: time="2026-06-29T10:43:19.778Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=13891176vm-test-run-nixbot-gitlab> gitlab # [   11.810867] gitaly[1389]: time="2026-06-29T10:43:19.778Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=13891177vm-test-run-nixbot-gitlab> gitlab # [   11.812064] gitaly[1389]: time="2026-06-29T10:43:19.778Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache235722046 clear_duration_ms=0 pid=1389 storage=default1178vm-test-run-nixbot-gitlab> gitlab # [   11.875877] gitaly[1389]: time="2026-06-29T10:43:19.849Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=13891179vm-test-run-nixbot-gitlab> gitlab # [   11.876876] gitaly[1389]: time="2026-06-29T10:43:19.849Z" level=info msg="counting repositories" pid=13891180vm-test-run-nixbot-gitlab> gitlab # [   11.877882] gitaly[1389]: time="2026-06-29T10:43:19.849Z" level=info msg="starting to count repositories" pid=1389 storage_path=/var/gitlab/state/repositories1181vm-test-run-nixbot-gitlab> gitlab # [   11.879210] gitaly[1389]: time="2026-06-29T10:43:19.849Z" level=info msg="finished tempdir cleaner walk" pid=1389 storage=default time_ms=01182vm-test-run-nixbot-gitlab> gitlab # [   11.880287] gitaly[1389]: time="2026-06-29T10:43:19.849Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=1389 schema=unix1183vm-test-run-nixbot-gitlab> gitlab # [   11.881344] gitaly[1389]: time="2026-06-29T10:43:19.849Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=1389 storage_path=/var/gitlab/state/repositories1184vm-test-run-nixbot-gitlab> gitlab # [   11.882436] gitaly[1389]: time="2026-06-29T10:43:19.849Z" level=info msg="completed counting all repositories" counting_duration=0 pid=13891185vm-test-run-nixbot-gitlab> gitlab # [   11.883836] gitaly[1389]: time="2026-06-29T10:43:19.849Z" level=info msg="listening at address" address=/run/gitaly/gitaly-1389/sock.d/intern pid=1389 schema=unix1186vm-test-run-nixbot-gitlab> gitlab # [   11.905088] gitaly[1389]: time="2026-06-29T10:43:19.878Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=13891187vm-test-run-nixbot-gitlab> gitlab # [   11.905871] gitaly[1389]: time="2026-06-29T10:43:19.878Z" level=info msg="Started Gitaly" duration_ms=294 pid=13891188vm-test-run-nixbot-gitlab> gitlab # [   11.910194] gitaly[1389]: time="2026-06-29T10:43:19.879Z" level=info msg="maintenance: daily scheduled" pid=1389 scheduled="2026-06-29 12:00:00 +0000 UTC"1189vm-test-run-nixbot-gitlab> gitlab # [   12.449491] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1393]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1190vm-test-run-nixbot-gitlab> gitlab # [   12.452449] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1393]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1191vm-test-run-nixbot-gitlab> gitlab # [   12.497263] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1393]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1192vm-test-run-nixbot-gitlab> gitlab # [   13.796584] gitaly[1389]: time="2026-06-29T10:43:21.769Z" level=info msg="License database preloaded" duration_ms=2184 pid=13891193vm-test-run-nixbot-gitlab> gitlab # [   14.701759] dhcpcd[1233]: eth0: leased 10.0.2.15 for 86400 seconds1194vm-test-run-nixbot-gitlab> gitlab # [   14.705615] dhcpcd[1233]: eth0: adding route to 10.0.2.0/241195vm-test-run-nixbot-gitlab> gitlab # [   14.708244] dhcpcd[1233]: eth0: adding default route via 10.0.2.21196vm-test-run-nixbot-gitlab> gitlab # [   14.839447] systemd[1]: Started DHCP Client.1197vm-test-run-nixbot-gitlab> gitlab # [   14.843715] systemd[1]: Reached target Network is Online.1198vm-test-run-nixbot-gitlab> gitlab # [   42.778964] postgres[1751]: [1751] ERROR:  relation "feature_gates" does not exist at character 5231199vm-test-run-nixbot-gitlab> gitlab # [   42.781682] postgres[1751]: [1751] STATEMENT:  SELECT a.attname, format_type(a.atttypid, a.atttypmod),1200vm-test-run-nixbot-gitlab> gitlab # [   42.783855] postgres[1751]: 	       pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1201vm-test-run-nixbot-gitlab> gitlab # [   42.786743] postgres[1751]: 	       c.collname, col_description(a.attrelid, a.attnum) AS comment,1202vm-test-run-nixbot-gitlab> gitlab # [   42.789503] postgres[1751]: 	       attidentity AS identity,1203vm-test-run-nixbot-gitlab> gitlab # [   42.792927] postgres[1751]: 	       attgenerated as attgenerated1204vm-test-run-nixbot-gitlab> gitlab # [   42.797051] postgres[1751]: 	  FROM pg_attribute a1205vm-test-run-nixbot-gitlab> gitlab # [   42.799811] postgres[1751]: 	  LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1206vm-test-run-nixbot-gitlab> gitlab # [   42.802581] postgres[1751]: 	  LEFT JOIN pg_type t ON a.atttypid = t.oid1207vm-test-run-nixbot-gitlab> gitlab # [   42.805930] postgres[1751]: 	  LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1208vm-test-run-nixbot-gitlab> gitlab # [   42.810761] postgres[1751]: 	 WHERE a.attrelid = '"feature_gates"'::regclass1209vm-test-run-nixbot-gitlab> gitlab # [   42.812788] postgres[1751]: 	   AND a.attnum > 0 AND NOT a.attisdropped1210vm-test-run-nixbot-gitlab> gitlab # [   42.814868] postgres[1751]: 	 ORDER BY a.attnum1211vm-test-run-nixbot-gitlab> gitlab # [   42.819049] postgres[1751]:1212vm-test-run-nixbot-gitlab> gitlab # [   42.999535] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1393]: W, [2026-06-29T10:43:50.972217 #1393]  WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1213vm-test-run-nixbot-gitlab> gitlab # [   48.288086] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1806]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE:  extension "btree_gist" already exists, skipping1214vm-test-run-nixbot-gitlab> gitlab # [   48.290742] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1806]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE:  extension "pg_trgm" already exists, skipping1215vm-test-run-nixbot-gitlab> gitlab # [   49.591389] gitlab-workhorse[1291]: time="2026-06-29T10:43:57Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1216vm-test-run-nixbot-gitlab> gitlab # [   59.602427] gitlab-workhorse[1291]: time="2026-06-29T10:44:07Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1217vm-test-run-nixbot-gitlab> gitlab # [   65.455199] postgres[1881]: [1881] ERROR:  relation "table_without_models" does not exist1218vm-test-run-nixbot-gitlab> gitlab # [   65.457598] postgres[1881]: [1881] STATEMENT:  SELECT pg_get_serial_sequence('table_without_models', 'id')1219vm-test-run-nixbot-gitlab> gitlab # [   69.603544] gitlab-workhorse[1291]: time="2026-06-29T10:44:17Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1220vm-test-run-nixbot-gitlab> gitlab # [   79.613501] gitlab-workhorse[1291]: time="2026-06-29T10:44:27Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1221vm-test-run-nixbot-gitlab> gitlab # [   79.894890] systemd[1]: Finished gitlab-db-config.service.1222vm-test-run-nixbot-gitlab> gitlab # [   79.902924] systemd[1]: Started gitlab-sidekiq.service.1223vm-test-run-nixbot-gitlab> gitlab # [   79.909607] systemd[1]: Starting gitlab.service...1224vm-test-run-nixbot-gitlab> gitlab # [   80.257648] bundle[2084]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1225vm-test-run-nixbot-gitlab> gitlab # [   80.266211] bundle[2084]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1226vm-test-run-nixbot-gitlab> gitlab # [   80.274625] bundle[2084]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1227vm-test-run-nixbot-gitlab> gitlab # [   83.104963] sidekiq-cluster[2083]: {"severity":"INFO","time":"2026-06-29T10:44:31.059Z","message":"Starting cluster with 1 processes"}1228vm-test-run-nixbot-gitlab> gitlab # [   83.448793] bundle[2084]: {"timestamp":"2026-06-29T10:44:31.396Z","pid":2084,"message":"* Enabling systemd notification integration"}1229vm-test-run-nixbot-gitlab> gitlab # [   83.455637] bundle[2084]: {"timestamp":"2026-06-29T10:44:31.421Z","pid":2084,"message":"Puma starting in cluster mode..."}1230vm-test-run-nixbot-gitlab> gitlab # [   83.461809] bundle[2084]: {"timestamp":"2026-06-29T10:44:31.421Z","pid":2084,"message":"* Puma version: 7.2.0 (\"On The Corner\")"}1231vm-test-run-nixbot-gitlab> gitlab # [   83.468600] bundle[2084]: {"timestamp":"2026-06-29T10:44:31.421Z","pid":2084,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1232vm-test-run-nixbot-gitlab> gitlab # [   83.476770] bundle[2084]: {"timestamp":"2026-06-29T10:44:31.421Z","pid":2084,"message":"*  Min threads: 0"}1233vm-test-run-nixbot-gitlab> gitlab # [   83.482550] bundle[2084]: {"timestamp":"2026-06-29T10:44:31.421Z","pid":2084,"message":"*  Max threads: 4"}1234vm-test-run-nixbot-gitlab> gitlab # [   83.488156] bundle[2084]: {"timestamp":"2026-06-29T10:44:31.421Z","pid":2084,"message":"*  Environment: production"}1235vm-test-run-nixbot-gitlab> gitlab # [   83.493983] bundle[2084]: {"timestamp":"2026-06-29T10:44:31.421Z","pid":2084,"message":"*   Master PID: 2084"}1236vm-test-run-nixbot-gitlab> gitlab # [   83.499776] bundle[2084]: {"timestamp":"2026-06-29T10:44:31.421Z","pid":2084,"message":"*      Workers: 2"}1237vm-test-run-nixbot-gitlab> gitlab # [   83.504253] bundle[2084]: {"timestamp":"2026-06-29T10:44:31.422Z","pid":2084,"message":"*     Restarts: (✔) hot (✖) phased (✖) refork"}1238vm-test-run-nixbot-gitlab> gitlab # [   83.509704] bundle[2084]: {"timestamp":"2026-06-29T10:44:31.422Z","pid":2084,"message":"* Preloading application"}1239vm-test-run-nixbot-gitlab> gitlab # [   89.623567] gitlab-workhorse[1291]: time="2026-06-29T10:44:37Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1240vm-test-run-nixbot-gitlab> gitlab # [   99.633770] gitlab-workhorse[1291]: time="2026-06-29T10:44:47Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1241vm-test-run-nixbot-gitlab> gitlab # [  107.912109] bundle[2084]: W, [2026-06-29T10:44:55.884827 #2084]  WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1242vm-test-run-nixbot-gitlab> gitlab # [  109.637796] gitlab-workhorse[1291]: time="2026-06-29T10:44:57Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1243vm-test-run-nixbot-gitlab> gitlab # [  109.937106] sidekiq-cluster[2114]: 2026-06-29T10:44:57.910Z pid=2114 tid=tze INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2114, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>429039616}1244vm-test-run-nixbot-gitlab> gitlab # [  110.124377] sidekiq-cluster[2114]: W, [2026-06-29T10:44:58.097750 #2114]  WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1245vm-test-run-nixbot-gitlab> gitlab # [  110.168230] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:44:58.141Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1246vm-test-run-nixbot-gitlab> gitlab # [  112.074950] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.047Z","message":"GitLab reliable fetch activated!"}1247vm-test-run-nixbot-gitlab> gitlab # [  112.077629] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.047Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1248vm-test-run-nixbot-gitlab> gitlab # [  112.109380] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.082Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1249vm-test-run-nixbot-gitlab> gitlab # [  112.118196] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.091Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1250vm-test-run-nixbot-gitlab> gitlab # [  112.126102] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.099Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1251vm-test-run-nixbot-gitlab> gitlab # [  112.136895] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.110Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1252vm-test-run-nixbot-gitlab> gitlab # [  112.147147] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.120Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1253vm-test-run-nixbot-gitlab> gitlab # [  112.156625] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.130Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1254vm-test-run-nixbot-gitlab> gitlab # [  112.165431] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.138Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1255vm-test-run-nixbot-gitlab> gitlab # [  112.173854] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.147Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1256vm-test-run-nixbot-gitlab> gitlab # [  112.182555] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.156Z","message":"Cron Jobs - added job with name expire_build_artifacts_worker in the namespace default"}1257vm-test-run-nixbot-gitlab> gitlab # [  112.192484] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.166Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1258vm-test-run-nixbot-gitlab> gitlab # [  112.201599] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.175Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1259vm-test-run-nixbot-gitlab> gitlab # [  112.211620] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.185Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1260vm-test-run-nixbot-gitlab> gitlab # [  112.221686] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.195Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1261vm-test-run-nixbot-gitlab> gitlab # [  112.230643] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.204Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1262vm-test-run-nixbot-gitlab> gitlab # [  112.239597] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.213Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1263vm-test-run-nixbot-gitlab> gitlab # [  112.248455] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.221Z","message":"Cron Jobs - added job with name environments_auto_delete_cron_worker in the namespace default"}1264vm-test-run-nixbot-gitlab> gitlab # [  112.257619] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.231Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1265vm-test-run-nixbot-gitlab> gitlab # [  112.268179] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.241Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1266vm-test-run-nixbot-gitlab> gitlab # [  112.277894] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.251Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1267vm-test-run-nixbot-gitlab> gitlab # [  112.287849] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.261Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1268vm-test-run-nixbot-gitlab> gitlab # [  112.297867] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.271Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1269vm-test-run-nixbot-gitlab> gitlab # [  112.308212] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.281Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1270vm-test-run-nixbot-gitlab> gitlab # [  112.318835] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.292Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1271vm-test-run-nixbot-gitlab> gitlab # [  112.328560] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.301Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1272vm-test-run-nixbot-gitlab> gitlab # [  112.337903] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.311Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1273vm-test-run-nixbot-gitlab> gitlab # [  112.347178] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.320Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1274vm-test-run-nixbot-gitlab> gitlab # [  112.356886] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.330Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1275vm-test-run-nixbot-gitlab> gitlab # [  112.368391] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.341Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1276vm-test-run-nixbot-gitlab> gitlab # [  112.377861] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.351Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1277vm-test-run-nixbot-gitlab> gitlab # [  112.387873] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.361Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1278vm-test-run-nixbot-gitlab> gitlab # [  112.399256] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.372Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1279vm-test-run-nixbot-gitlab> gitlab # [  112.408138] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.381Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1280vm-test-run-nixbot-gitlab> gitlab # [  112.418494] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.392Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1281vm-test-run-nixbot-gitlab> gitlab # [  112.427874] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.401Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1282vm-test-run-nixbot-gitlab> gitlab # [  112.437861] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.411Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1283vm-test-run-nixbot-gitlab> gitlab # [  112.446950] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.420Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1284vm-test-run-nixbot-gitlab> gitlab # [  112.455278] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.428Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1285vm-test-run-nixbot-gitlab> gitlab # [  112.463981] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.437Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1286vm-test-run-nixbot-gitlab> gitlab # [  112.473143] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.446Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1287vm-test-run-nixbot-gitlab> gitlab # [  112.481477] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.455Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1288vm-test-run-nixbot-gitlab> gitlab # [  112.489846] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.463Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1289vm-test-run-nixbot-gitlab> gitlab # [  112.499205] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.472Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1290vm-test-run-nixbot-gitlab> gitlab # [  112.507817] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.481Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1291vm-test-run-nixbot-gitlab> gitlab # [  112.516927] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.490Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1292vm-test-run-nixbot-gitlab> gitlab # [  112.526210] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.499Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1293vm-test-run-nixbot-gitlab> gitlab # [  112.536188] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.509Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1294vm-test-run-nixbot-gitlab> gitlab # [  112.546208] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.519Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1295vm-test-run-nixbot-gitlab> gitlab # [  112.554862] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.528Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1296vm-test-run-nixbot-gitlab> gitlab # [  112.564429] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.537Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1297vm-test-run-nixbot-gitlab> gitlab # [  112.574205] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.547Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1298vm-test-run-nixbot-gitlab> gitlab # [  112.583368] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.556Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1299vm-test-run-nixbot-gitlab> gitlab # [  112.592632] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.566Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1300vm-test-run-nixbot-gitlab> gitlab # [  112.602650] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.576Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1301vm-test-run-nixbot-gitlab> gitlab # [  112.626746] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.600Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1302vm-test-run-nixbot-gitlab> gitlab # [  112.636244] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.609Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1303vm-test-run-nixbot-gitlab> gitlab # [  112.644552] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.618Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1304vm-test-run-nixbot-gitlab> gitlab # [  112.652874] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.626Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1305vm-test-run-nixbot-gitlab> gitlab # [  112.663958] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.637Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1306vm-test-run-nixbot-gitlab> gitlab # [  112.672613] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.646Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1307vm-test-run-nixbot-gitlab> gitlab # [  112.679316] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.652Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1308vm-test-run-nixbot-gitlab> gitlab # [  112.687180] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.660Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1309vm-test-run-nixbot-gitlab> gitlab # [  112.694695] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.668Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1310vm-test-run-nixbot-gitlab> gitlab # [  112.701436] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.675Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1311vm-test-run-nixbot-gitlab> gitlab # [  112.709543] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.683Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1312vm-test-run-nixbot-gitlab> gitlab # [  112.717231] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.690Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1313vm-test-run-nixbot-gitlab> gitlab # [  112.723960] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.697Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1314vm-test-run-nixbot-gitlab> gitlab # [  112.731231] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.704Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1315vm-test-run-nixbot-gitlab> gitlab # [  112.758691] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.732Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1316vm-test-run-nixbot-gitlab> gitlab # [  112.762130] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.735Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1317vm-test-run-nixbot-gitlab> gitlab # [  112.765721] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.739Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1318vm-test-run-nixbot-gitlab> gitlab # [  112.770699] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.744Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1319vm-test-run-nixbot-gitlab> gitlab # [  112.775809] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.749Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1320vm-test-run-nixbot-gitlab> gitlab # [  112.782305] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.755Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1321vm-test-run-nixbot-gitlab> gitlab # [  112.786708] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.760Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1322vm-test-run-nixbot-gitlab> gitlab # [  112.790175] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.763Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1323vm-test-run-nixbot-gitlab> gitlab # [  112.793214] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.766Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1324vm-test-run-nixbot-gitlab> gitlab # [  112.796564] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.770Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1325vm-test-run-nixbot-gitlab> gitlab # [  112.799639] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.773Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1326vm-test-run-nixbot-gitlab> gitlab # [  112.803636] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.777Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1327vm-test-run-nixbot-gitlab> gitlab # [  112.806879] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.780Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1328vm-test-run-nixbot-gitlab> gitlab # [  112.810612] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.784Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1329vm-test-run-nixbot-gitlab> gitlab # [  112.813899] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.787Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1330vm-test-run-nixbot-gitlab> gitlab # [  112.818251] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.792Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1331vm-test-run-nixbot-gitlab> gitlab # [  112.821629] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.795Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1332vm-test-run-nixbot-gitlab> gitlab # [  112.824921] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.798Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1333vm-test-run-nixbot-gitlab> gitlab # [  112.828132] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.801Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1334vm-test-run-nixbot-gitlab> gitlab # [  112.831592] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.805Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1335vm-test-run-nixbot-gitlab> gitlab # [  112.835282] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.808Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1336vm-test-run-nixbot-gitlab> gitlab # [  112.838852] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.812Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1337vm-test-run-nixbot-gitlab> gitlab # [  112.841772] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.815Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1338vm-test-run-nixbot-gitlab> gitlab # [  112.845332] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.819Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1339vm-test-run-nixbot-gitlab> gitlab # [  112.848153] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.821Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1340vm-test-run-nixbot-gitlab> gitlab # [  112.850936] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.824Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1341vm-test-run-nixbot-gitlab> gitlab # [  112.854680] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.828Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1342vm-test-run-nixbot-gitlab> gitlab # [  112.857981] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.831Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1343vm-test-run-nixbot-gitlab> gitlab # [  112.861831] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.835Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1344vm-test-run-nixbot-gitlab> gitlab # [  112.865474] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.839Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1345vm-test-run-nixbot-gitlab> gitlab # [  112.868591] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.842Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1346vm-test-run-nixbot-gitlab> gitlab # [  112.872124] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.845Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1347vm-test-run-nixbot-gitlab> gitlab # [  112.883049] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.856Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1348vm-test-run-nixbot-gitlab> gitlab # [  112.886237] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.860Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1349vm-test-run-nixbot-gitlab> gitlab # [  112.889218] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.862Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1350vm-test-run-nixbot-gitlab> gitlab # [  112.893103] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.866Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1351vm-test-run-nixbot-gitlab> gitlab # [  112.896386] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.870Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1352vm-test-run-nixbot-gitlab> gitlab # [  112.899977] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.873Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1353vm-test-run-nixbot-gitlab> gitlab # [  112.903579] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.877Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1354vm-test-run-nixbot-gitlab> gitlab # [  112.906775] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.880Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1355vm-test-run-nixbot-gitlab> gitlab # [  112.909835] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.883Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1356vm-test-run-nixbot-gitlab> gitlab # [  112.913893] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.887Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1357vm-test-run-nixbot-gitlab> gitlab # [  112.917076] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:00.890Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1358vm-test-run-nixbot-gitlab> gitlab # [  119.647708] gitlab-workhorse[1291]: time="2026-06-29T10:45:07Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1359vm-test-run-nixbot-gitlab> gitlab # [  129.649824] gitlab-workhorse[1291]: time="2026-06-29T10:45:17Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1360vm-test-run-nixbot-gitlab> gitlab # [  134.165823] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:22.138Z","message":"Booted Rails 7.2.3 application in production environment"}1361vm-test-run-nixbot-gitlab> gitlab # [  134.168132] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:22.139Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1362vm-test-run-nixbot-gitlab> gitlab # [  134.169725] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:22.139Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1363vm-test-run-nixbot-gitlab> gitlab # [  134.171442] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:22.139Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1364vm-test-run-nixbot-gitlab> gitlab # [  134.179872] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:22.153Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1365vm-test-run-nixbot-gitlab> gitlab # [  135.450160] bundle[2084]: {"timestamp":"2026-06-29T10:45:23.422Z","pid":2084,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1366vm-test-run-nixbot-gitlab> gitlab # [  135.464764] bundle[2084]: {"timestamp":"2026-06-29T10:45:23.423Z","pid":2084,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1367vm-test-run-nixbot-gitlab> gitlab # [  135.471927] bundle[2084]: {"timestamp":"2026-06-29T10:45:23.423Z","pid":2084,"message":"! #\u003cThread:0x00007256c4e9d328 /nix/store/g5qraa51hgg5fl7n3da89ygf00qfv95x-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/g5qraa51hgg5fl7n3da89ygf00qfv95x-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1368vm-test-run-nixbot-gitlab> gitlab # [  135.490402] bundle[2084]: {"timestamp":"2026-06-29T10:45:23.423Z","pid":2084,"message":"! #\u003cThread:0x00007256c50ee578 /nix/store/wg5r0zmy3rh1kga7z4z7mbs298p0wxli-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/wg5r0zmy3rh1kga7z4z7mbs298p0wxli-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1369vm-test-run-nixbot-gitlab> gitlab # [  135.492504] bundle[2084]: {"timestamp":"2026-06-29T10:45:23.423Z","pid":2084,"message":"Use Ctrl-C to stop"}1370vm-test-run-nixbot-gitlab> gitlab # [  135.494664] bundle[2084]: {"timestamp":"2026-06-29T10:45:23.423Z","pid":2084,"message":"Sending status to systemd every 1.0 sec"}1371vm-test-run-nixbot-gitlab> gitlab # [  135.497293] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:23.428Z","message":"Cleaning working queues"}1372vm-test-run-nixbot-gitlab> gitlab # [  135.498109] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:23.433Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"7235098b6edceb3145ab950e","created_at":"2026-06-29T10:44:27.277Z","trace_propagation_headers":{"sentry-trace":"4c2b5f33be7942979028d400184dd43d-60c91f2776934351","baggage":"sentry-trace_id=4c2b5f33be7942979028d400184dd43d,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"0e8c5d91b70d7839dd9fd4bae6a9c144","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-29T10:44:27.287Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-7235098b6edceb3145ab950e: start","job_status":"start","queue_duration_s":56.145035,"scheduling_latency_s":56.145143}1373vm-test-run-nixbot-gitlab> gitlab # [  135.612364] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:23.585Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"7235098b6edceb3145ab950e","created_at":"2026-06-29T10:44:27.277Z","trace_propagation_headers":{"sentry-trace":"4c2b5f33be7942979028d400184dd43d-60c91f2776934351","baggage":"sentry-trace_id=4c2b5f33be7942979028d400184dd43d,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"0e8c5d91b70d7839dd9fd4bae6a9c144","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-29T10:44:27.287Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-7235098b6edceb3145ab950e: done: 0.15291 sec","job_status":"done","queue_duration_s":56.145035,"scheduling_latency_s":56.145143,"redis_calls":26,"redis_duration_s":0.0067729999999999995,"redis_read_bytes":914,"redis_write_bytes":2887,"redis_feature_flag_calls":15,"redis_feature_flag_duration_s":0.003751,"redis_feature_flag_read_bytes":904,"redis_feature_flag_write_bytes":1640,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001934,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001088,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.064135,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.070313406,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.15291,"completed_at":"2026-06-29T10:45:23.585Z","load_balancing_strategy":"primary","db_duration_s":0.041041,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1374vm-test-run-nixbot-gitlab> gitlab # [  135.741379] bundle[2084]: {"timestamp":"2026-06-29T10:45:23.714Z","pid":2084,"message":"- Worker 0 (PID: 2621) booted in 0.25s, phase: 0"}1375vm-test-run-nixbot-gitlab> gitlab # [  135.854370] bundle[2084]: {"timestamp":"2026-06-29T10:45:23.827Z","pid":2084,"message":"- Worker 1 (PID: 2626) booted in 0.33s, phase: 0"}1376vm-test-run-nixbot-gitlab> gitlab # [  135.862125] systemd[1]: Started gitlab.service.1377vm-test-run-nixbot-gitlab> gitlab # [  135.865505] systemd[1]: Reached target Common target for all GitLab services..1378vm-test-run-nixbot-gitlab> gitlab # [  135.870135] systemd[1]: Starting setup-gitlab.service...1379vm-test-run-nixbot-gitlab> gitlab # [  135.893699] setup-gitlab-start[2651]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1380vm-test-run-nixbot-gitlab> gitlab # [  138.288268] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:45:26 +0000] "GET /users/sign_in HTTP/1.1" 200 10624 "" "curl/8.20.0" 23401381vm-test-run-nixbot-gitlab> gitlab # [  138.297235] setup-gitlab-start[2674]: ++ curl -fs -X POST -H 'Content-Type: application/json' -d '{"grant_type":"password","username":"root","password":"notproduction"}' http://gitlab/oauth/token1382vm-test-run-nixbot-gitlab> gitlab # [  138.299509] setup-gitlab-start[2675]: ++ jq -r .access_token1383vm-test-run-nixbot-gitlab> gitlab # [  139.114394] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:45:27 +0000] "POST /oauth/token HTTP/1.1" 200 244 "" "curl/8.20.0" 7891384vm-test-run-nixbot-gitlab> gitlab # [  139.120850] setup-gitlab-start[2651]: + BEARER=4d02a22efa6e41be75d30815dc81239bb7cbcca6cbf6d06438eaa5a7bd14df091385vm-test-run-nixbot-gitlab> gitlab # [  139.123633] setup-gitlab-start[2651]: + AUTH='Authorization: Bearer 4d02a22efa6e41be75d30815dc81239bb7cbcca6cbf6d06438eaa5a7bd14df09'1386vm-test-run-nixbot-gitlab> gitlab # [  139.125622] setup-gitlab-start[2651]: + curl -fs -X PUT -H 'Authorization: Bearer 4d02a22efa6e41be75d30815dc81239bb7cbcca6cbf6d06438eaa5a7bd14df09' 'http://gitlab/api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true'1387vm-test-run-nixbot-gitlab> gitlab # [  140.531845] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:45:28 +0000] "PUT /api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true HTTP/1.1" 200 17133 "" "curl/8.20.0" 13841388vm-test-run-nixbot-gitlab> gitlab # [  140.543937] setup-gitlab-start[2651]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner Rails.cache.clear1389vm-test-run-nixbot-gitlab> gitlab # [  140.560177] runuser[2699]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1390vm-test-run-nixbot-gitlab> gitlab # [  140.934925] setup-gitlab-start[2700]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1391vm-test-run-nixbot-gitlab> gitlab # [  140.943782] setup-gitlab-start[2700]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1392vm-test-run-nixbot-gitlab> gitlab # [  140.952552] setup-gitlab-start[2700]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1393vm-test-run-nixbot-gitlab> gitlab # [  141.512255] setup-gitlab-start[2700]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1394vm-test-run-nixbot-gitlab> gitlab # [  141.521187] setup-gitlab-start[2700]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1395vm-test-run-nixbot-gitlab> gitlab # [  141.529697] setup-gitlab-start[2700]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1396vm-test-run-nixbot-gitlab> gitlab # [  146.463444] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:34.435Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"e6869afc96a637a62c1737d9","created_at":"2026-06-29T10:45:34.415Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ada2e0dcdbf60cdaaf92394cceae9ba7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-29T10:45:34.432Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-e6869afc96a637a62c1737d9: start","job_status":"start","queue_duration_s":0.003458,"scheduling_latency_s":0.003525}1397vm-test-run-nixbot-gitlab> gitlab # [  146.579591] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:34.552Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"e6869afc96a637a62c1737d9","created_at":"2026-06-29T10:45:34.415Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ada2e0dcdbf60cdaaf92394cceae9ba7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-29T10:45:34.432Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-e6869afc96a637a62c1737d9: done: 0.117503 sec","job_status":"done","queue_duration_s":0.003458,"scheduling_latency_s":0.003525,"redis_calls":10,"redis_duration_s":0.0064269999999999996,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000915,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004434,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001078,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035068,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001368331,"duration_s":0.117503,"completed_at":"2026-06-29T10:45:34.552Z","load_balancing_strategy":"primary","db_duration_s":0.027568,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1398vm-test-run-nixbot-gitlab> gitlab # [  146.640955] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:34.559Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"2bd2540bc93d9fa8e65392d1","created_at":"2026-06-29T10:45:34.456Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ae9a47fe4d1d92c6ff7b9e131d4d40b6","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:34.494Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-2bd2540bc93d9fa8e65392d1: start","job_status":"start","queue_duration_s":0.064397,"scheduling_latency_s":0.06442}1399vm-test-run-nixbot-gitlab> gitlab # [  146.738501] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:34.711Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"2bd2540bc93d9fa8e65392d1","created_at":"2026-06-29T10:45:34.456Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ae9a47fe4d1d92c6ff7b9e131d4d40b6","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:34.494Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-2bd2540bc93d9fa8e65392d1: done: 0.152357 sec","job_status":"done","queue_duration_s":0.064397,"scheduling_latency_s":0.06442,"redis_calls":16,"redis_duration_s":0.019079,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.009471,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005937,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003671,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.045818,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000737245,"duration_s":0.152357,"completed_at":"2026-06-29T10:45:34.711Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.076866,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1400vm-test-run-nixbot-gitlab> gitlab # [  146.812219] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:34.785Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Issues::RescheduleStuckIssueRebalancesWorker","jid":"70e4f92d2ddea1376970efca","created_at":"2026-06-29T10:45:34.503Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3e22293fef0822de5caaa4264cdafdac","meta.root_caller_id":"Cronjob","meta.feature_category":"team_planning","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fc2fa70b35c572764e1a8fbcbc4fdd3df48ec5a5855870d65915560d23915134","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:34.513Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Issues::RescheduleStuckIssueRebalancesWorker JID-70e4f92d2ddea1376970efca: start","job_status":"start","queue_duration_s":0.271603,"scheduling_latency_s":0.271642}1401vm-test-run-nixbot-gitlab> gitlab # [  146.918158] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:34.890Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Issues::RescheduleStuckIssueRebalancesWorker","jid":"70e4f92d2ddea1376970efca","created_at":"2026-06-29T10:45:34.503Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3e22293fef0822de5caaa4264cdafdac","meta.root_caller_id":"Cronjob","meta.feature_category":"team_planning","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fc2fa70b35c572764e1a8fbcbc4fdd3df48ec5a5855870d65915560d23915134","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:34.513Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Issues::RescheduleStuckIssueRebalancesWorker JID-70e4f92d2ddea1376970efca: done: 0.105284 sec","job_status":"done","queue_duration_s":0.271603,"scheduling_latency_s":0.271642,"redis_calls":11,"redis_duration_s":0.017121,"redis_read_bytes":9,"redis_write_bytes":1239,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00212,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.009881,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":610,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00512,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":298,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025734,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.003066871,"duration_s":0.105284,"completed_at":"2026-06-29T10:45:34.890Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015348,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1402vm-test-run-nixbot-gitlab> gitlab # [  146.931626] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:34.904Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"0fcd7888130fc6d913b1df16","created_at":"2026-06-29T10:45:34.531Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9efd292bf512edab1e46b33b630d2cf7","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:34.547Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-0fcd7888130fc6d913b1df16: start","job_status":"start","queue_duration_s":0.356727,"scheduling_latency_s":0.356784}1403vm-test-run-nixbot-gitlab> gitlab # [  147.018541] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:34.955Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"0fcd7888130fc6d913b1df16","created_at":"2026-06-29T10:45:34.531Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9efd292bf512edab1e46b33b630d2cf7","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:34.547Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-0fcd7888130fc6d913b1df16: done: 0.050842 sec","job_status":"done","queue_duration_s":0.356727,"scheduling_latency_s":0.356784,"redis_calls":13,"redis_duration_s":0.00468,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000994,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002151,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001535,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027375,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.015576561,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.050842,"completed_at":"2026-06-29T10:45:34.955Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006087,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1404vm-test-run-nixbot-gitlab> gitlab # [  147.021478] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:34.957Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"03a1444203a5176f3bce29bb","created_at":"2026-06-29T10:45:34.558Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"56a887f64a27a9763b150be0aebcd3cc","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:34.565Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-03a1444203a5176f3bce29bb: start","job_status":"start","queue_duration_s":0.391773,"scheduling_latency_s":0.391795}1405vm-test-run-nixbot-gitlab> gitlab # [  147.044433] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:34.981Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"03a1444203a5176f3bce29bb","created_at":"2026-06-29T10:45:34.558Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"56a887f64a27a9763b150be0aebcd3cc","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:34.565Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-03a1444203a5176f3bce29bb: done: 0.024267 sec","job_status":"done","queue_duration_s":0.391773,"scheduling_latency_s":0.391795,"redis_calls":12,"redis_duration_s":0.002866,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000428,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.0014,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001038,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011959,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000539454,"exclusive_lock_hold_duration_s":0.007046147000000001,"duration_s":0.024267,"completed_at":"2026-06-29T10:45:34.981Z","load_balancing_strategy":"primary","db_duration_s":0.005227,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1406vm-test-run-nixbot-gitlab> gitlab # [  147.147506] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:34.982Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"a31f54d94f8c9ec6a1cad7b8","created_at":"2026-06-29T10:45:34.573Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"69e35ae24692c95947810b6cb5d4d454","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:34.580Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-a31f54d94f8c9ec6a1cad7b8: start","job_status":"start","queue_duration_s":0.401877,"scheduling_latency_s":0.401896}1407vm-test-run-nixbot-gitlab> gitlab # [  147.149311] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:35.002Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"a31f54d94f8c9ec6a1cad7b8","created_at":"2026-06-29T10:45:34.573Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"69e35ae24692c95947810b6cb5d4d454","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:34.580Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-a31f54d94f8c9ec6a1cad7b8: done: 0.019336 sec","job_status":"done","queue_duration_s":0.401877,"scheduling_latency_s":0.401896,"redis_calls":12,"redis_duration_s":0.002921,"redis_read_bytes":11,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000869,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001563,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000489,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011466,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000352838,"duration_s":0.019336,"completed_at":"2026-06-29T10:45:35.002Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000764,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1408vm-test-run-nixbot-gitlab> gitlab # [  147.238521] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:35.003Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckMergeJobsWorker","jid":"321416a7fd233b92997e7f14","created_at":"2026-06-29T10:45:34.584Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"72a88c2760039cf2174e48e3d8d04de9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:ce304458c2d63d5d84855b65f51692704db52bf763a718b10c08a2bd5f588f06","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:34.618Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckMergeJobsWorker JID-321416a7fd233b92997e7f14: start","job_status":"start","queue_duration_s":0.385225,"scheduling_latency_s":0.385245}1409vm-test-run-nixbot-gitlab> gitlab # [  147.240507] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:35.024Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckMergeJobsWorker","jid":"321416a7fd233b92997e7f14","created_at":"2026-06-29T10:45:34.584Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"72a88c2760039cf2174e48e3d8d04de9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:ce304458c2d63d5d84855b65f51692704db52bf763a718b10c08a2bd5f588f06","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:34.618Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckMergeJobsWorker JID-321416a7fd233b92997e7f14: done: 0.021035 sec","job_status":"done","queue_duration_s":0.385225,"scheduling_latency_s":0.385245,"redis_calls":11,"redis_duration_s":0.002719,"redis_read_bytes":10,"redis_write_bytes":1022,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000455,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":283,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001452,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000812,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":225,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011808,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000387479,"duration_s":0.021035,"completed_at":"2026-06-29T10:45:35.024Z","load_balancing_strategy":"primary","db_duration_s":0.002196,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1410vm-test-run-nixbot-gitlab> gitlab # [  147.290057] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:35.026Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"f3801e39649e938085d7b20c","created_at":"2026-06-29T10:45:34.623Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e4e4cd0ee2d34fc5e44d465f07241c68","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:34.627Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-f3801e39649e938085d7b20c: start","job_status":"start","queue_duration_s":0.398503,"scheduling_latency_s":0.398523}1411vm-test-run-nixbot-gitlab> gitlab # [  147.842971] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:35.815Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"f3801e39649e938085d7b20c","created_at":"2026-06-29T10:45:34.623Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e4e4cd0ee2d34fc5e44d465f07241c68","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:34.627Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-f3801e39649e938085d7b20c: done: 0.789494 sec","job_status":"done","queue_duration_s":0.398503,"scheduling_latency_s":0.398523,"redis_calls":14,"redis_duration_s":0.007492,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.003203,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001966,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002323,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.34,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.324,"db_ci_duration_s":0.017,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.231067,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001631772,"exclusive_lock_hold_duration_s":0.435399471,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.789494,"completed_at":"2026-06-29T10:45:35.815Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.344321,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1412vm-test-run-nixbot-gitlab> gitlab # [  147.915247] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:35.888Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"4c586dbca76e263d7b29a1d3","created_at":"2026-06-29T10:45:34.642Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"63e2e2d9ec3250ca21da94c3ace91020","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:34.646Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-4c586dbca76e263d7b29a1d3: start","job_status":"start","queue_duration_s":1.241665,"scheduling_latency_s":1.2417}1413vm-test-run-nixbot-gitlab> gitlab # [  147.986862] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:35.960Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"4c586dbca76e263d7b29a1d3","created_at":"2026-06-29T10:45:34.642Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"63e2e2d9ec3250ca21da94c3ace91020","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:34.646Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-4c586dbca76e263d7b29a1d3: done: 0.07169 sec","job_status":"done","queue_duration_s":1.241665,"scheduling_latency_s":1.2417,"redis_calls":10,"redis_duration_s":0.006346,"redis_read_bytes":9,"redis_write_bytes":1215,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001659,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003529,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001158,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027374,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000960736,"duration_s":0.07169,"completed_at":"2026-06-29T10:45:35.959Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001149,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1414vm-test-run-nixbot-gitlab> gitlab # [  148.024372] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:35.963Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"cf012e09088a79df043142f1","created_at":"2026-06-29T10:45:34.649Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8e177daabd7a67b50222ccd6087f3390","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:34.653Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-cf012e09088a79df043142f1: start","job_status":"start","queue_duration_s":1.309203,"scheduling_latency_s":1.309327}1415vm-test-run-nixbot-gitlab> gitlab # [  148.138821] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:36.111Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"cf012e09088a79df043142f1","created_at":"2026-06-29T10:45:34.649Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8e177daabd7a67b50222ccd6087f3390","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:34.653Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-cf012e09088a79df043142f1: done: 0.148224 sec","job_status":"done","queue_duration_s":1.309203,"scheduling_latency_s":1.309327,"redis_calls":12,"redis_duration_s":0.006697,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000984,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003048,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002665,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.059,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.059,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041032,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001224178,"exclusive_lock_hold_duration_s":0.023512205999999997,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.148224,"completed_at":"2026-06-29T10:45:36.111Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.060114,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1416vm-test-run-nixbot-gitlab> gitlab # [  148.228916] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:36.202Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"85e8d060eb00902b3c51b439","created_at":"2026-06-29T10:45:34.661Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e63901d53eea3aeec14163faae998c63","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:34.666Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-85e8d060eb00902b3c51b439: start","job_status":"start","queue_duration_s":1.535789,"scheduling_latency_s":1.53595}1417vm-test-run-nixbot-gitlab> gitlab # [  148.259185] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:36.232Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1418vm-test-run-nixbot-gitlab> gitlab # [  148.276525] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:36.249Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"85e8d060eb00902b3c51b439","created_at":"2026-06-29T10:45:34.661Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e63901d53eea3aeec14163faae998c63","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:34.666Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-85e8d060eb00902b3c51b439: done: 0.047288 sec","job_status":"done","queue_duration_s":1.535789,"scheduling_latency_s":1.53595,"redis_calls":10,"redis_duration_s":0.0072829999999999995,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001155,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004874,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001254,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025362,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000942019,"duration_s":0.047288,"completed_at":"2026-06-29T10:45:36.249Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001178,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1419vm-test-run-nixbot-gitlab> gitlab # [  148.279429] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:36.252Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainVerificationCronWorker","jid":"03410f26e2d547bfccfcbaaa","created_at":"2026-06-29T10:45:34.676Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e5d478d6bc5d6d52139b1aac2f44d314","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:4b3095972298b58b9c376f502255d63e0934f66e5921dfe9547414a5f347be92","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:34.688Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainVerificationCronWorker JID-03410f26e2d547bfccfcbaaa: start","job_status":"start","queue_duration_s":1.564622,"scheduling_latency_s":1.564662}1420vm-test-run-nixbot-gitlab> gitlab # [  148.311832] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:36.285Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainVerificationCronWorker","jid":"03410f26e2d547bfccfcbaaa","created_at":"2026-06-29T10:45:34.676Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e5d478d6bc5d6d52139b1aac2f44d314","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:4b3095972298b58b9c376f502255d63e0934f66e5921dfe9547414a5f347be92","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:34.688Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainVerificationCronWorker JID-03410f26e2d547bfccfcbaaa: done: 0.032571 sec","job_status":"done","queue_duration_s":1.564622,"scheduling_latency_s":1.564662,"redis_calls":10,"redis_duration_s":0.003166,"redis_read_bytes":9,"redis_write_bytes":1097,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000477,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":309,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002088,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000601,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":218,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014933,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000515428,"duration_s":0.032571,"completed_at":"2026-06-29T10:45:36.285Z","load_balancing_strategy":"primary","db_duration_s":0.007369,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1421vm-test-run-nixbot-gitlab> gitlab # [  148.411134] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:36.286Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"1a55156f98958f4078ed3717","created_at":"2026-06-29T10:45:34.696Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c9dc1e216ea7e63f1182b26d8b2828b2","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:34.705Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-1a55156f98958f4078ed3717: start","job_status":"start","queue_duration_s":1.58147,"scheduling_latency_s":1.58149}1422vm-test-run-nixbot-gitlab> gitlab # [  148.412301] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:36.312Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"1a55156f98958f4078ed3717","created_at":"2026-06-29T10:45:34.696Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c9dc1e216ea7e63f1182b26d8b2828b2","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:34.705Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-1a55156f98958f4078ed3717: done: 0.026212 sec","job_status":"done","queue_duration_s":1.58147,"scheduling_latency_s":1.58149,"redis_calls":12,"redis_duration_s":0.0028970000000000003,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000482,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001359,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001056,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011188,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000477435,"exclusive_lock_hold_duration_s":0.005724191,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.026212,"completed_at":"2026-06-29T10:45:36.312Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007646,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1423vm-test-run-nixbot-gitlab> gitlab # [  148.491141] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:36.314Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"a032084cd9b70a0fdec9cc33","created_at":"2026-06-29T10:45:34.782Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6dd443435966e996686d4d1a60957a81","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:34.817Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-a032084cd9b70a0fdec9cc33: start","job_status":"start","queue_duration_s":1.496502,"scheduling_latency_s":1.496521}1424vm-test-run-nixbot-gitlab> gitlab # [  148.492525] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:36.324Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1425vm-test-run-nixbot-gitlab> gitlab # [  148.493909] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:36.331Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"a032084cd9b70a0fdec9cc33","created_at":"2026-06-29T10:45:34.782Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6dd443435966e996686d4d1a60957a81","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:34.817Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-a032084cd9b70a0fdec9cc33: done: 0.017183 sec","job_status":"done","queue_duration_s":1.496502,"scheduling_latency_s":1.496521,"redis_calls":10,"redis_duration_s":0.002908,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000513,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001773,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000622,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009447,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00043134,"duration_s":0.017183,"completed_at":"2026-06-29T10:45:36.331Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000408,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1426vm-test-run-nixbot-gitlab> gitlab # [  148.581302] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:36.333Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"ee82f2a63d50632add0e78e1","created_at":"2026-06-29T10:45:34.859Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7adadbfa0b140aeddcf4cad9af89cfcf","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:34.876Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-ee82f2a63d50632add0e78e1: start","job_status":"start","queue_duration_s":1.456417,"scheduling_latency_s":1.456437}1427vm-test-run-nixbot-gitlab> gitlab # [  148.789200] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:36.762Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"ee82f2a63d50632add0e78e1","created_at":"2026-06-29T10:45:34.859Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7adadbfa0b140aeddcf4cad9af89cfcf","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:34.876Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-ee82f2a63d50632add0e78e1: done: 0.429138 sec","job_status":"done","queue_duration_s":1.456417,"scheduling_latency_s":1.456437,"redis_calls":574,"redis_duration_s":0.141335,"redis_read_bytes":573,"redis_write_bytes":49866,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000779,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002134,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.138422,"redis_shared_state_read_bytes":567,"redis_shared_state_write_bytes":49017,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.327942,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000648965,"duration_s":0.429138,"completed_at":"2026-06-29T10:45:36.762Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000596,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1428vm-test-run-nixbot-gitlab> gitlab # [  148.793980] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:36.763Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"bd5d399873ef46b9d65d8c2d","created_at":"2026-06-29T10:45:34.897Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a2e83a59f041dfd99b4cf02e4ef21b8e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:35.369Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-bd5d399873ef46b9d65d8c2d: start","job_status":"start","queue_duration_s":1.39465,"scheduling_latency_s":1.394672}1429vm-test-run-nixbot-gitlab> gitlab # [  148.896902] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:36.775Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1430vm-test-run-nixbot-gitlab> gitlab # [  148.897747] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:36.781Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"bd5d399873ef46b9d65d8c2d","created_at":"2026-06-29T10:45:34.897Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a2e83a59f041dfd99b4cf02e4ef21b8e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:35.369Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-bd5d399873ef46b9d65d8c2d: done: 0.017865 sec","job_status":"done","queue_duration_s":1.39465,"scheduling_latency_s":1.394672,"redis_calls":10,"redis_duration_s":0.002408,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000439,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001463,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000506,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009934,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000384965,"duration_s":0.017865,"completed_at":"2026-06-29T10:45:36.781Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000534,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1431vm-test-run-nixbot-gitlab> gitlab # [  148.977443] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:36.783Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"78b5c46d0cc75c644f6f3ec9","created_at":"2026-06-29T10:45:35.397Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9d8ebbc66e6523265ca84be57440d8b1","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:35.424Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-78b5c46d0cc75c644f6f3ec9: start","job_status":"start","queue_duration_s":1.359061,"scheduling_latency_s":1.359081}1432vm-test-run-nixbot-gitlab> gitlab # [  148.978540] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:36.803Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"78b5c46d0cc75c644f6f3ec9","created_at":"2026-06-29T10:45:35.397Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9d8ebbc66e6523265ca84be57440d8b1","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:35.424Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-78b5c46d0cc75c644f6f3ec9: done: 0.019911 sec","job_status":"done","queue_duration_s":1.359061,"scheduling_latency_s":1.359081,"redis_calls":11,"redis_duration_s":0.004001,"redis_read_bytes":9,"redis_write_bytes":1174,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000705,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002025,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000612,"redis_rate_limiting_write_bytes":51,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000659,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01135,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000500343,"extra.cells_stale_requests_cleanup_cron_worker.removed_count":0,"duration_s":0.019911,"completed_at":"2026-06-29T10:45:36.803Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00058,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1433vm-test-run-nixbot-gitlab> gitlab # [  149.007321] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:36.804Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"706a6d7f7e351650f5ec0964","created_at":"2026-06-29T10:45:35.448Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9a959b5f800f713f4e8a412e2903af49","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:35.460Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-706a6d7f7e351650f5ec0964: start","job_status":"start","queue_duration_s":1.343979,"scheduling_latency_s":1.343999}1434vm-test-run-nixbot-gitlab> gitlab # [  149.029711] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:36.985Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"706a6d7f7e351650f5ec0964","created_at":"2026-06-29T10:45:35.448Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9a959b5f800f713f4e8a412e2903af49","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:35.460Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-706a6d7f7e351650f5ec0964: done: 0.181171 sec","job_status":"done","queue_duration_s":1.343979,"scheduling_latency_s":1.343999,"redis_calls":12,"redis_duration_s":0.004112,"redis_read_bytes":11,"redis_write_bytes":1370,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000623,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001882,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001607,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":124,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":124,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.032,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":123,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":80,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.032,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.127495,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000846476,"exclusive_lock_hold_duration_s":0.151043219,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.181171,"completed_at":"2026-06-29T10:45:36.985Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.055734,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1435vm-test-run-nixbot-gitlab> gitlab # [  149.104363] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:36.987Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"ea05ef2a1e66bd2d46033fd0","created_at":"2026-06-29T10:45:35.479Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"16252a8ef52b3fdccb60427876ead7c7","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:35.490Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-ea05ef2a1e66bd2d46033fd0: start","job_status":"start","queue_duration_s":1.497014,"scheduling_latency_s":1.497036}1436vm-test-run-nixbot-gitlab> gitlab # [  149.177950] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.016Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"ea05ef2a1e66bd2d46033fd0","created_at":"2026-06-29T10:45:35.479Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"16252a8ef52b3fdccb60427876ead7c7","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:35.490Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-ea05ef2a1e66bd2d46033fd0: done: 0.029327 sec","job_status":"done","queue_duration_s":1.497014,"scheduling_latency_s":1.497036,"redis_calls":10,"redis_duration_s":0.002364,"redis_read_bytes":9,"redis_write_bytes":1071,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000462,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001443,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000459,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014584,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000341663,"duration_s":0.029327,"completed_at":"2026-06-29T10:45:37.016Z","load_balancing_strategy":"primary","db_duration_s":0.007622,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1437vm-test-run-nixbot-gitlab> gitlab # [  149.183278] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.018Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"ed5dd65522c2e35eb3143fef","created_at":"2026-06-29T10:45:35.504Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dde298399b4f34700a78c5c03740db3f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:35.516Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-ed5dd65522c2e35eb3143fef: start","job_status":"start","queue_duration_s":1.501728,"scheduling_latency_s":1.501758}1438vm-test-run-nixbot-gitlab> gitlab # [  149.257562] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.028Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1439vm-test-run-nixbot-gitlab> gitlab # [  149.258899] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.034Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"ed5dd65522c2e35eb3143fef","created_at":"2026-06-29T10:45:35.504Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dde298399b4f34700a78c5c03740db3f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:35.516Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-ed5dd65522c2e35eb3143fef: done: 0.01633 sec","job_status":"done","queue_duration_s":1.501728,"scheduling_latency_s":1.501758,"redis_calls":10,"redis_duration_s":0.002404,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000495,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001395,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000514,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009566,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00036066,"duration_s":0.01633,"completed_at":"2026-06-29T10:45:37.034Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000338,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1440vm-test-run-nixbot-gitlab> gitlab # [  149.277925] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.035Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"83fa469f09778ae107e963b4","created_at":"2026-06-29T10:45:35.563Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"51ac5a639f4792319628aab8d4284ac6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:35.572Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-83fa469f09778ae107e963b4: start","job_status":"start","queue_duration_s":1.463302,"scheduling_latency_s":1.463321}1441vm-test-run-nixbot-gitlab> gitlab # [  149.299790] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.062Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"83fa469f09778ae107e963b4","created_at":"2026-06-29T10:45:35.563Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"51ac5a639f4792319628aab8d4284ac6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:35.572Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-83fa469f09778ae107e963b4: done: 0.027085 sec","job_status":"done","queue_duration_s":1.463302,"scheduling_latency_s":1.463321,"redis_calls":19,"redis_duration_s":0.004059,"redis_read_bytes":21,"redis_write_bytes":4785,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000807,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000296,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002306,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00065,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018303,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000342502,"duration_s":0.027085,"completed_at":"2026-06-29T10:45:37.062Z","load_balancing_strategy":"primary","db_duration_s":0.001426,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1442vm-test-run-nixbot-gitlab> gitlab # [  149.407109] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.064Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"2b33e13ef0eb5a5bc8b7d3f4","created_at":"2026-06-29T10:45:35.582Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:35.592Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-2b33e13ef0eb5a5bc8b7d3f4: start","job_status":"start","queue_duration_s":1.471248,"scheduling_latency_s":1.471269}1443vm-test-run-nixbot-gitlab> gitlab # [  149.408611] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.160Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"2b33e13ef0eb5a5bc8b7d3f4","created_at":"2026-06-29T10:45:35.582Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:35.592Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-2b33e13ef0eb5a5bc8b7d3f4: done: 0.096476 sec","job_status":"done","queue_duration_s":1.471248,"scheduling_latency_s":1.471269,"redis_calls":71,"redis_duration_s":0.020012,"redis_read_bytes":120,"redis_write_bytes":42624,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002638,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000639,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.015746,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000989,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.072726,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000810717,"duration_s":0.096476,"completed_at":"2026-06-29T10:45:37.160Z","load_balancing_strategy":"primary","db_duration_s":0.00422,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1444vm-test-run-nixbot-gitlab> gitlab # [  149.486995] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.164Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"8e6b286c8c2bc9e7809b0ef4","created_at":"2026-06-29T10:45:35.600Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"baa2a2aae32c5ab35f723ce66174abc4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-29T10:45:35.607Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-8e6b286c8c2bc9e7809b0ef4: start","job_status":"start","queue_duration_s":1.556728,"scheduling_latency_s":1.55675}1445vm-test-run-nixbot-gitlab> gitlab # [  149.488178] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.175Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1446vm-test-run-nixbot-gitlab> gitlab # [  149.489285] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.181Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"8e6b286c8c2bc9e7809b0ef4","created_at":"2026-06-29T10:45:35.600Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"baa2a2aae32c5ab35f723ce66174abc4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-29T10:45:35.607Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-8e6b286c8c2bc9e7809b0ef4: done: 0.01727 sec","job_status":"done","queue_duration_s":1.556728,"scheduling_latency_s":1.55675,"redis_calls":10,"redis_duration_s":0.002301,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000456,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00129,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000555,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010144,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000341105,"duration_s":0.01727,"completed_at":"2026-06-29T10:45:37.181Z","load_balancing_strategy":"primary","db_duration_s":0.000496,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1447vm-test-run-nixbot-gitlab> gitlab # [  149.587042] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.182Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"d630074bb39c8ac9de40e357","created_at":"2026-06-29T10:45:35.612Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f6b9f4636095632663cf2ebdd233ebf8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-29T10:45:35.617Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-d630074bb39c8ac9de40e357: start","job_status":"start","queue_duration_s":1.564896,"scheduling_latency_s":1.564918}1448vm-test-run-nixbot-gitlab> gitlab # [  149.588218] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.192Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1449vm-test-run-nixbot-gitlab> gitlab # [  149.589759] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.199Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"d630074bb39c8ac9de40e357","created_at":"2026-06-29T10:45:35.612Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f6b9f4636095632663cf2ebdd233ebf8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-29T10:45:35.617Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-d630074bb39c8ac9de40e357: done: 0.016298 sec","job_status":"done","queue_duration_s":1.564896,"scheduling_latency_s":1.564918,"redis_calls":10,"redis_duration_s":0.002151,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000407,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001302,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000442,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009458,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000355911,"duration_s":0.016298,"completed_at":"2026-06-29T10:45:37.198Z","load_balancing_strategy":"primary","db_duration_s":0.00038,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1450vm-test-run-nixbot-gitlab> gitlab # [  149.696242] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.200Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"8e2191d6539dcce32fd079bd","created_at":"2026-06-29T10:45:35.626Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e9535dd11e493fb493447f2aca3f3b7b","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:35.631Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-8e2191d6539dcce32fd079bd: start","job_status":"start","queue_duration_s":1.568519,"scheduling_latency_s":1.568538}1451vm-test-run-nixbot-gitlab> gitlab # [  149.697820] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.219Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"8e2191d6539dcce32fd079bd","created_at":"2026-06-29T10:45:35.626Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e9535dd11e493fb493447f2aca3f3b7b","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:35.631Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-8e2191d6539dcce32fd079bd: done: 0.019211 sec","job_status":"done","queue_duration_s":1.568519,"scheduling_latency_s":1.568538,"redis_calls":10,"redis_duration_s":0.00233,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000401,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001478,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000451,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01113,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000357308,"duration_s":0.019211,"completed_at":"2026-06-29T10:45:37.219Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00176,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1452vm-test-run-nixbot-gitlab> gitlab # [  149.776956] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.221Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"43281d33c781dee79c3bc419","created_at":"2026-06-29T10:45:35.637Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3306ac4281f340eea691d963ca88f9fc","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:35.643Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-43281d33c781dee79c3bc419: start","job_status":"start","queue_duration_s":1.577564,"scheduling_latency_s":1.577583}1453vm-test-run-nixbot-gitlab> gitlab # [  149.778189] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.256Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"43281d33c781dee79c3bc419","created_at":"2026-06-29T10:45:35.637Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3306ac4281f340eea691d963ca88f9fc","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-29T10:45:35.643Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-43281d33c781dee79c3bc419: done: 0.035562 sec","job_status":"done","queue_duration_s":1.577564,"scheduling_latency_s":1.577583,"redis_calls":10,"redis_duration_s":0.006430999999999999,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002489,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00339,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000552,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01156,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000534984,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.035562,"completed_at":"2026-06-29T10:45:37.256Z","load_balancing_strategy":"primary","db_duration_s":0.013632,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1454vm-test-run-nixbot-gitlab> gitlab # [  149.819223] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.260Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9eac55be3e3a601b17dd83ea","created_at":"2026-06-29T10:45:37.048Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-ab53f9d6291c4513","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"51ac5a639f4792319628aab8d4284ac6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.056Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9eac55be3e3a601b17dd83ea: start","job_status":"start","queue_duration_s":0.203392,"scheduling_latency_s":0.203413}1455vm-test-run-nixbot-gitlab> gitlab # [  149.897929] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.281Z","message":"No existing merge request to be cleaned up."}1456vm-test-run-nixbot-gitlab> gitlab # [  149.899568] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.292Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9eac55be3e3a601b17dd83ea","created_at":"2026-06-29T10:45:37.048Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-ab53f9d6291c4513","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"51ac5a639f4792319628aab8d4284ac6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.056Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9eac55be3e3a601b17dd83ea: done: 0.03244 sec","job_status":"done","queue_duration_s":0.203392,"scheduling_latency_s":0.203413,"redis_calls":13,"redis_duration_s":0.007594999999999999,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001107,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002574,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003914,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014595,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000384407,"duration_s":0.03244,"completed_at":"2026-06-29T10:45:37.292Z","load_balancing_strategy":"primary","db_duration_s":0.007879,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1457vm-test-run-nixbot-gitlab> gitlab # [  149.905284] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.294Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"12256ff057cfe17a02e33a34","created_at":"2026-06-29T10:45:37.048Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-ab53f9d6291c4513","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"51ac5a639f4792319628aab8d4284ac6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.056Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-12256ff057cfe17a02e33a34: start","job_status":"start","queue_duration_s":0.237168,"scheduling_latency_s":0.237189}1458vm-test-run-nixbot-gitlab> gitlab # [  149.925040] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.305Z","message":"No existing merge request to be cleaned up."}1459vm-test-run-nixbot-gitlab> gitlab # [  149.928188] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.312Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"12256ff057cfe17a02e33a34","created_at":"2026-06-29T10:45:37.048Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-ab53f9d6291c4513","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"51ac5a639f4792319628aab8d4284ac6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.056Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-12256ff057cfe17a02e33a34: done: 0.017948 sec","job_status":"done","queue_duration_s":0.237168,"scheduling_latency_s":0.237189,"redis_calls":10,"redis_duration_s":0.002339,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000903,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001436,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010257,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017948,"completed_at":"2026-06-29T10:45:37.312Z","load_balancing_strategy":"primary","db_duration_s":0.003414,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1460vm-test-run-nixbot-gitlab> gitlab # [  150.011243] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.313Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f6b0af353c53c5122122c206","created_at":"2026-06-29T10:45:37.048Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-ab53f9d6291c4513","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"51ac5a639f4792319628aab8d4284ac6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.056Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f6b0af353c53c5122122c206: start","job_status":"start","queue_duration_s":0.256417,"scheduling_latency_s":0.256438}1461vm-test-run-nixbot-gitlab> gitlab # [  150.012791] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.323Z","message":"No existing merge request to be cleaned up."}1462vm-test-run-nixbot-gitlab> gitlab # [  150.013986] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.329Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f6b0af353c53c5122122c206","created_at":"2026-06-29T10:45:37.048Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-ab53f9d6291c4513","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"51ac5a639f4792319628aab8d4284ac6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.056Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f6b0af353c53c5122122c206: done: 0.015919 sec","job_status":"done","queue_duration_s":0.256417,"scheduling_latency_s":0.256438,"redis_calls":10,"redis_duration_s":0.0028929999999999997,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001019,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001874,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009807,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015919,"completed_at":"2026-06-29T10:45:37.329Z","load_balancing_strategy":"primary","db_duration_s":0.001552,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1463vm-test-run-nixbot-gitlab> gitlab # [  150.088258] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.330Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4c6f7f3c4cafb01b27bd999e","created_at":"2026-06-29T10:45:37.048Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-ab53f9d6291c4513","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"51ac5a639f4792319628aab8d4284ac6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.056Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4c6f7f3c4cafb01b27bd999e: start","job_status":"start","queue_duration_s":0.273627,"scheduling_latency_s":0.273647}1464vm-test-run-nixbot-gitlab> gitlab # [  150.089306] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.340Z","message":"No existing merge request to be cleaned up."}1465vm-test-run-nixbot-gitlab> gitlab # [  150.091437] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.345Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4c6f7f3c4cafb01b27bd999e","created_at":"2026-06-29T10:45:37.048Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-ab53f9d6291c4513","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"51ac5a639f4792319628aab8d4284ac6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.056Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4c6f7f3c4cafb01b27bd999e: done: 0.015315 sec","job_status":"done","queue_duration_s":0.273627,"scheduling_latency_s":0.273647,"redis_calls":10,"redis_duration_s":0.002175,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000855,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00132,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009982,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015315,"completed_at":"2026-06-29T10:45:37.345Z","load_balancing_strategy":"primary","db_duration_s":0.001486,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1466vm-test-run-nixbot-gitlab> gitlab # [  150.160205] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.347Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"500f704338a48d274336ab0b","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-500f704338a48d274336ab0b: start","job_status":"start","queue_duration_s":0.198119,"scheduling_latency_s":0.19814}1467vm-test-run-nixbot-gitlab> gitlab # [  150.171223] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.368Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"500f704338a48d274336ab0b","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-500f704338a48d274336ab0b: done: 0.020935 sec","job_status":"done","queue_duration_s":0.198119,"scheduling_latency_s":0.19814,"redis_calls":16,"redis_duration_s":0.003488,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000413,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001058,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002017,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013153,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000349207,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020935,"completed_at":"2026-06-29T10:45:37.368Z","load_balancing_strategy":"primary","db_duration_s":0.000346,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1468vm-test-run-nixbot-gitlab> gitlab # [  150.237259] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.369Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c5802a9f5a8a50fa8fb280d","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c5802a9f5a8a50fa8fb280d: start","job_status":"start","queue_duration_s":0.220433,"scheduling_latency_s":0.220453}1469vm-test-run-nixbot-gitlab> gitlab # [  150.257252] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.387Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c5802a9f5a8a50fa8fb280d","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c5802a9f5a8a50fa8fb280d: done: 0.01791 sec","job_status":"done","queue_duration_s":0.220433,"scheduling_latency_s":0.220453,"redis_calls":13,"redis_duration_s":0.00553,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001407,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004123,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010164,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01791,"completed_at":"2026-06-29T10:45:37.387Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1470vm-test-run-nixbot-gitlab> gitlab # [  150.328091] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.388Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2231a025ad225fe5f0561ced","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2231a025ad225fe5f0561ced: start","job_status":"start","queue_duration_s":0.239643,"scheduling_latency_s":0.239663}1471vm-test-run-nixbot-gitlab> gitlab # [  150.338107] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.403Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2231a025ad225fe5f0561ced","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2231a025ad225fe5f0561ced: done: 0.015205 sec","job_status":"done","queue_duration_s":0.239643,"scheduling_latency_s":0.239663,"redis_calls":13,"redis_duration_s":0.002758,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000875,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001883,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01006,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015205,"completed_at":"2026-06-29T10:45:37.403Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1472vm-test-run-nixbot-gitlab> gitlab # [  150.408173] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.405Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f583e90fa187b498e7edd52","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f583e90fa187b498e7edd52: start","job_status":"start","queue_duration_s":0.256156,"scheduling_latency_s":0.256176}1473vm-test-run-nixbot-gitlab> gitlab # [  150.418989] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.420Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f583e90fa187b498e7edd52","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f583e90fa187b498e7edd52: done: 0.015511 sec","job_status":"done","queue_duration_s":0.256156,"scheduling_latency_s":0.256176,"redis_calls":13,"redis_duration_s":0.003052,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000952,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0021,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010221,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015511,"completed_at":"2026-06-29T10:45:37.420Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1474vm-test-run-nixbot-gitlab> gitlab # [  150.481807] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.422Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04652c3168493e4edcb9659d","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04652c3168493e4edcb9659d: start","job_status":"start","queue_duration_s":0.273067,"scheduling_latency_s":0.273088}1475vm-test-run-nixbot-gitlab> gitlab # [  150.563343] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.436Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04652c3168493e4edcb9659d","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04652c3168493e4edcb9659d: done: 0.014808 sec","job_status":"done","queue_duration_s":0.273067,"scheduling_latency_s":0.273088,"redis_calls":13,"redis_duration_s":0.002732,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000848,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001884,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009983,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014808,"completed_at":"2026-06-29T10:45:37.436Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1476vm-test-run-nixbot-gitlab> gitlab # [  150.565165] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.438Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"957a5638805bfc9373a7fd4c","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-957a5638805bfc9373a7fd4c: start","job_status":"start","queue_duration_s":0.289306,"scheduling_latency_s":0.289327}1477vm-test-run-nixbot-gitlab> gitlab # [  150.647590] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.453Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"957a5638805bfc9373a7fd4c","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-957a5638805bfc9373a7fd4c: done: 0.014963 sec","job_status":"done","queue_duration_s":0.289306,"scheduling_latency_s":0.289327,"redis_calls":13,"redis_duration_s":0.002792,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00087,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001922,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010033,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014963,"completed_at":"2026-06-29T10:45:37.453Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1478vm-test-run-nixbot-gitlab> gitlab # [  150.649976] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.454Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e0ebb47f79f2690f04254a05","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e0ebb47f79f2690f04254a05: start","job_status":"start","queue_duration_s":0.30562,"scheduling_latency_s":0.305642}1479vm-test-run-nixbot-gitlab> gitlab # [  150.728075] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.473Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e0ebb47f79f2690f04254a05","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e0ebb47f79f2690f04254a05: done: 0.018365 sec","job_status":"done","queue_duration_s":0.30562,"scheduling_latency_s":0.305642,"redis_calls":13,"redis_duration_s":0.004285,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001119,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003166,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011698,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018365,"completed_at":"2026-06-29T10:45:37.472Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1480vm-test-run-nixbot-gitlab> gitlab # [  150.729497] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.474Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee1bf855203a2e6978b43747","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee1bf855203a2e6978b43747: start","job_status":"start","queue_duration_s":0.325565,"scheduling_latency_s":0.325591}1481vm-test-run-nixbot-gitlab> gitlab # [  150.802528] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.490Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee1bf855203a2e6978b43747","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee1bf855203a2e6978b43747: done: 0.015822 sec","job_status":"done","queue_duration_s":0.325565,"scheduling_latency_s":0.325591,"redis_calls":13,"redis_duration_s":0.003126,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001013,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002113,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009912,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015822,"completed_at":"2026-06-29T10:45:37.490Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1482vm-test-run-nixbot-gitlab> gitlab # [  150.804439] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.491Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d08d06a78e2821615da97d37","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d08d06a78e2821615da97d37: start","job_status":"start","queue_duration_s":0.342765,"scheduling_latency_s":0.342786}1483vm-test-run-nixbot-gitlab> gitlab # [  150.876569] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.507Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d08d06a78e2821615da97d37","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d08d06a78e2821615da97d37: done: 0.015507 sec","job_status":"done","queue_duration_s":0.342765,"scheduling_latency_s":0.342786,"redis_calls":13,"redis_duration_s":0.003107,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000896,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002211,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009935,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015507,"completed_at":"2026-06-29T10:45:37.507Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1484vm-test-run-nixbot-gitlab> gitlab # [  150.884227] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.508Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8acce1cd28fa52423d01bec0","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8acce1cd28fa52423d01bec0: start","job_status":"start","queue_duration_s":0.359714,"scheduling_latency_s":0.359735}1485vm-test-run-nixbot-gitlab> gitlab # [  150.957718] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.524Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8acce1cd28fa52423d01bec0","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8acce1cd28fa52423d01bec0: done: 0.015374 sec","job_status":"done","queue_duration_s":0.359714,"scheduling_latency_s":0.359735,"redis_calls":13,"redis_duration_s":0.003018,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000848,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00217,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010193,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015374,"completed_at":"2026-06-29T10:45:37.524Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1486vm-test-run-nixbot-gitlab> gitlab # [  150.964053] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.525Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f7b873548969dcc14adbf1f7","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f7b873548969dcc14adbf1f7: start","job_status":"start","queue_duration_s":0.376499,"scheduling_latency_s":0.376521}1487vm-test-run-nixbot-gitlab> gitlab # [  151.040176] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.540Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f7b873548969dcc14adbf1f7","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f7b873548969dcc14adbf1f7: done: 0.015168 sec","job_status":"done","queue_duration_s":0.376499,"scheduling_latency_s":0.376521,"redis_calls":13,"redis_duration_s":0.002757,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000811,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001946,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010068,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015168,"completed_at":"2026-06-29T10:45:37.540Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1488vm-test-run-nixbot-gitlab> gitlab # [  151.044396] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.541Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15cad3880da8d941746a692e","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15cad3880da8d941746a692e: start","job_status":"start","queue_duration_s":0.392753,"scheduling_latency_s":0.392774}1489vm-test-run-nixbot-gitlab> gitlab # [  151.120859] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.556Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15cad3880da8d941746a692e","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15cad3880da8d941746a692e: done: 0.014918 sec","job_status":"done","queue_duration_s":0.392753,"scheduling_latency_s":0.392774,"redis_calls":13,"redis_duration_s":0.002936,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000846,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00209,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010001,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014918,"completed_at":"2026-06-29T10:45:37.556Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1490vm-test-run-nixbot-gitlab> gitlab # [  151.124999] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.558Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6905f40a60890d5ffb10354","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6905f40a60890d5ffb10354: start","job_status":"start","queue_duration_s":0.408941,"scheduling_latency_s":0.408963}1491vm-test-run-nixbot-gitlab> gitlab # [  151.198298] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.576Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6905f40a60890d5ffb10354","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6905f40a60890d5ffb10354: done: 0.018788 sec","job_status":"done","queue_duration_s":0.408941,"scheduling_latency_s":0.408963,"redis_calls":13,"redis_duration_s":0.004163999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001022,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003142,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011551,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018788,"completed_at":"2026-06-29T10:45:37.576Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1492vm-test-run-nixbot-gitlab> gitlab # [  151.206120] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.578Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bbe706a73b3d4b18e2025482","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bbe706a73b3d4b18e2025482: start","job_status":"start","queue_duration_s":0.429239,"scheduling_latency_s":0.429266}1493vm-test-run-nixbot-gitlab> gitlab # [  151.225791] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.594Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bbe706a73b3d4b18e2025482","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bbe706a73b3d4b18e2025482: done: 0.015975 sec","job_status":"done","queue_duration_s":0.429239,"scheduling_latency_s":0.429266,"redis_calls":13,"redis_duration_s":0.0030470000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000893,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002154,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009829,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015975,"completed_at":"2026-06-29T10:45:37.594Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1494vm-test-run-nixbot-gitlab> gitlab # [  151.310230] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.595Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1acbee350b62bcf74d06008e","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1acbee350b62bcf74d06008e: start","job_status":"start","queue_duration_s":0.446616,"scheduling_latency_s":0.446638}1495vm-test-run-nixbot-gitlab> gitlab # [  151.311910] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.611Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1acbee350b62bcf74d06008e","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1acbee350b62bcf74d06008e: done: 0.015434 sec","job_status":"done","queue_duration_s":0.446616,"scheduling_latency_s":0.446638,"redis_calls":13,"redis_duration_s":0.002864,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000922,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001942,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009692,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015434,"completed_at":"2026-06-29T10:45:37.611Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1496vm-test-run-nixbot-gitlab> gitlab # [  151.384973] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.612Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b0f92cb51eb8cf44e34945e2","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b0f92cb51eb8cf44e34945e2: start","job_status":"start","queue_duration_s":0.463427,"scheduling_latency_s":0.463449}1497vm-test-run-nixbot-gitlab> gitlab # [  151.386916] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.627Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b0f92cb51eb8cf44e34945e2","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b0f92cb51eb8cf44e34945e2: done: 0.015106 sec","job_status":"done","queue_duration_s":0.463427,"scheduling_latency_s":0.463449,"redis_calls":13,"redis_duration_s":0.00284,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000924,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001916,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010092,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015106,"completed_at":"2026-06-29T10:45:37.627Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1498vm-test-run-nixbot-gitlab> gitlab # [  151.467321] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.629Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5070bded24f1541ed6a5c24","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5070bded24f1541ed6a5c24: start","job_status":"start","queue_duration_s":0.47998,"scheduling_latency_s":0.480001}1499vm-test-run-nixbot-gitlab> gitlab # [  151.468416] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.644Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5070bded24f1541ed6a5c24","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5070bded24f1541ed6a5c24: done: 0.015894 sec","job_status":"done","queue_duration_s":0.47998,"scheduling_latency_s":0.480001,"redis_calls":13,"redis_duration_s":0.003561,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000861,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0027,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010082,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015894,"completed_at":"2026-06-29T10:45:37.644Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1500vm-test-run-nixbot-gitlab> gitlab # [  151.537968] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.646Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6101912212513df786322926","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6101912212513df786322926: start","job_status":"start","queue_duration_s":0.497359,"scheduling_latency_s":0.497381}1501vm-test-run-nixbot-gitlab> gitlab # [  151.608320] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.661Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6101912212513df786322926","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6101912212513df786322926: done: 0.015194 sec","job_status":"done","queue_duration_s":0.497359,"scheduling_latency_s":0.497381,"redis_calls":13,"redis_duration_s":0.002901,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001892,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010156,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015194,"completed_at":"2026-06-29T10:45:37.661Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1502vm-test-run-nixbot-gitlab> gitlab # [  151.610298] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.663Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"11abdf4682abcc0e5af0f3df","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-11abdf4682abcc0e5af0f3df: start","job_status":"start","queue_duration_s":0.514049,"scheduling_latency_s":0.514069}1503vm-test-run-nixbot-gitlab> gitlab # [  151.628771] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.679Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"11abdf4682abcc0e5af0f3df","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-11abdf4682abcc0e5af0f3df: done: 0.016343 sec","job_status":"done","queue_duration_s":0.514049,"scheduling_latency_s":0.514069,"redis_calls":13,"redis_duration_s":0.004124,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000861,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003263,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010037,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016343,"completed_at":"2026-06-29T10:45:37.679Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1504vm-test-run-nixbot-gitlab> gitlab # [  151.748939] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.680Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47c95435a850ae6a19be4ab2","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47c95435a850ae6a19be4ab2: start","job_status":"start","queue_duration_s":0.531844,"scheduling_latency_s":0.531907}1505vm-test-run-nixbot-gitlab> gitlab # [  151.750306] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.696Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47c95435a850ae6a19be4ab2","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47c95435a850ae6a19be4ab2: done: 0.015537 sec","job_status":"done","queue_duration_s":0.531844,"scheduling_latency_s":0.531907,"redis_calls":13,"redis_duration_s":0.0029890000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00093,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002059,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009872,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015537,"completed_at":"2026-06-29T10:45:37.696Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1506vm-test-run-nixbot-gitlab> gitlab # [  151.772452] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.697Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a21a4c4bfa47e41b8d8f81dc","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a21a4c4bfa47e41b8d8f81dc: start","job_status":"start","queue_duration_s":0.548813,"scheduling_latency_s":0.548834}1507vm-test-run-nixbot-gitlab> gitlab # [  151.792516] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.713Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a21a4c4bfa47e41b8d8f81dc","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a21a4c4bfa47e41b8d8f81dc: done: 0.015356 sec","job_status":"done","queue_duration_s":0.548813,"scheduling_latency_s":0.548834,"redis_calls":13,"redis_duration_s":0.002875,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000924,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001951,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009782,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015356,"completed_at":"2026-06-29T10:45:37.713Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1508vm-test-run-nixbot-gitlab> gitlab # [  151.896901] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.714Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"54e5f7c6c5bd4348c5dfa2af","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-54e5f7c6c5bd4348c5dfa2af: start","job_status":"start","queue_duration_s":0.565428,"scheduling_latency_s":0.565449}1509vm-test-run-nixbot-gitlab> gitlab # [  151.898445] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.730Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"54e5f7c6c5bd4348c5dfa2af","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-54e5f7c6c5bd4348c5dfa2af: done: 0.015703 sec","job_status":"done","queue_duration_s":0.565428,"scheduling_latency_s":0.565449,"redis_calls":13,"redis_duration_s":0.003339,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001259,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00208,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009812,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015703,"completed_at":"2026-06-29T10:45:37.730Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1510vm-test-run-nixbot-gitlab> gitlab # [  151.980835] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.731Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"88ec65758c10fe3adf1d0122","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-88ec65758c10fe3adf1d0122: start","job_status":"start","queue_duration_s":0.58241,"scheduling_latency_s":0.582431}1511vm-test-run-nixbot-gitlab> gitlab # [  151.982136] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.746Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"88ec65758c10fe3adf1d0122","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-88ec65758c10fe3adf1d0122: done: 0.015425 sec","job_status":"done","queue_duration_s":0.58241,"scheduling_latency_s":0.582431,"redis_calls":13,"redis_duration_s":0.002942,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00092,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002022,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009848,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015425,"completed_at":"2026-06-29T10:45:37.746Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1512vm-test-run-nixbot-gitlab> gitlab # [  152.051216] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.748Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2a7889aa17fcafdf21a4f4f3","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2a7889aa17fcafdf21a4f4f3: start","job_status":"start","queue_duration_s":0.59932,"scheduling_latency_s":0.59934}1513vm-test-run-nixbot-gitlab> gitlab # [  152.052542] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.764Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2a7889aa17fcafdf21a4f4f3","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2a7889aa17fcafdf21a4f4f3: done: 0.015689 sec","job_status":"done","queue_duration_s":0.59932,"scheduling_latency_s":0.59934,"redis_calls":13,"redis_duration_s":0.002958,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000942,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002016,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009716,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015689,"completed_at":"2026-06-29T10:45:37.764Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1514vm-test-run-nixbot-gitlab> gitlab # [  152.124244] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.765Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7fc9972110b56c24dd473b83","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7fc9972110b56c24dd473b83: start","job_status":"start","queue_duration_s":0.616304,"scheduling_latency_s":0.616326}1515vm-test-run-nixbot-gitlab> gitlab # [  152.125819] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.780Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7fc9972110b56c24dd473b83","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7fc9972110b56c24dd473b83: done: 0.015517 sec","job_status":"done","queue_duration_s":0.616304,"scheduling_latency_s":0.616326,"redis_calls":13,"redis_duration_s":0.00308,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001045,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002035,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009751,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015517,"completed_at":"2026-06-29T10:45:37.780Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1516vm-test-run-nixbot-gitlab> gitlab # [  152.212397] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.782Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"282602c004dded5eeaecca06","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-282602c004dded5eeaecca06: start","job_status":"start","queue_duration_s":0.63325,"scheduling_latency_s":0.63327}1517vm-test-run-nixbot-gitlab> gitlab # [  152.214057] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.798Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"282602c004dded5eeaecca06","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-282602c004dded5eeaecca06: done: 0.016378 sec","job_status":"done","queue_duration_s":0.63325,"scheduling_latency_s":0.63327,"redis_calls":13,"redis_duration_s":0.003202,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000874,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002328,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009813,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016378,"completed_at":"2026-06-29T10:45:37.798Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1518vm-test-run-nixbot-gitlab> gitlab # [  152.284189] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.800Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2885e69020509d018ff7016f","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2885e69020509d018ff7016f: start","job_status":"start","queue_duration_s":0.651046,"scheduling_latency_s":0.651066}1519vm-test-run-nixbot-gitlab> gitlab # [  152.285342] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.815Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2885e69020509d018ff7016f","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2885e69020509d018ff7016f: done: 0.015346 sec","job_status":"done","queue_duration_s":0.651046,"scheduling_latency_s":0.651066,"redis_calls":13,"redis_duration_s":0.002989,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000872,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002117,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009619,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015346,"completed_at":"2026-06-29T10:45:37.815Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1520vm-test-run-nixbot-gitlab> gitlab # [  152.369948] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.816Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a33f7d6b3c71a368eee8562","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a33f7d6b3c71a368eee8562: start","job_status":"start","queue_duration_s":0.667784,"scheduling_latency_s":0.667804}1521vm-test-run-nixbot-gitlab> gitlab # [  152.371076] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.832Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a33f7d6b3c71a368eee8562","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a33f7d6b3c71a368eee8562: done: 0.015691 sec","job_status":"done","queue_duration_s":0.667784,"scheduling_latency_s":0.667804,"redis_calls":13,"redis_duration_s":0.003303,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000872,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002431,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009713,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015691,"completed_at":"2026-06-29T10:45:37.832Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1522vm-test-run-nixbot-gitlab> gitlab # [  152.456284] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:37.833Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c1b172d226d7e7713252572","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c1b172d226d7e7713252572: start","job_status":"start","queue_duration_s":0.684785,"scheduling_latency_s":0.684805}1523vm-test-run-nixbot-gitlab> gitlab # [  152.457894] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.092Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c1b172d226d7e7713252572","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c1b172d226d7e7713252572: done: 0.25898 sec","job_status":"done","queue_duration_s":0.684785,"scheduling_latency_s":0.684805,"redis_calls":13,"redis_duration_s":0.0032979999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00111,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002188,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010877,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.25898,"completed_at":"2026-06-29T10:45:38.092Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1524vm-test-run-nixbot-gitlab> gitlab # [  152.533203] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.094Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cfc4a0f7c300db1004ef933b","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cfc4a0f7c300db1004ef933b: start","job_status":"start","queue_duration_s":0.945449,"scheduling_latency_s":0.945476}1525vm-test-run-nixbot-gitlab> gitlab # [  152.534396] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.110Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cfc4a0f7c300db1004ef933b","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cfc4a0f7c300db1004ef933b: done: 0.016023 sec","job_status":"done","queue_duration_s":0.945449,"scheduling_latency_s":0.945476,"redis_calls":13,"redis_duration_s":0.0027530000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00084,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001913,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010656,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016023,"completed_at":"2026-06-29T10:45:38.110Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1526vm-test-run-nixbot-gitlab> gitlab # [  152.610184] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.111Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"331a9ff08f1d025617a29d9d","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-331a9ff08f1d025617a29d9d: start","job_status":"start","queue_duration_s":0.96282,"scheduling_latency_s":0.962841}1527vm-test-run-nixbot-gitlab> gitlab # [  152.611573] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.127Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"331a9ff08f1d025617a29d9d","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-331a9ff08f1d025617a29d9d: done: 0.015166 sec","job_status":"done","queue_duration_s":0.96282,"scheduling_latency_s":0.962841,"redis_calls":13,"redis_duration_s":0.002853,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000969,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001884,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010106,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015166,"completed_at":"2026-06-29T10:45:38.126Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1528vm-test-run-nixbot-gitlab> gitlab # [  152.693328] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.128Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bef0560fa1af89d0d2ecba71","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bef0560fa1af89d0d2ecba71: start","job_status":"start","queue_duration_s":0.979349,"scheduling_latency_s":0.979369}1529vm-test-run-nixbot-gitlab> gitlab # [  152.694443] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.150Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bef0560fa1af89d0d2ecba71","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bef0560fa1af89d0d2ecba71: done: 0.021959 sec","job_status":"done","queue_duration_s":0.979349,"scheduling_latency_s":0.979369,"redis_calls":13,"redis_duration_s":0.007507,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001679,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005828,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012035,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021959,"completed_at":"2026-06-29T10:45:38.150Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1530vm-test-run-nixbot-gitlab> gitlab # [  152.744124] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.152Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b030f1dc864379b0fda5db11","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b030f1dc864379b0fda5db11: start","job_status":"start","queue_duration_s":1.002934,"scheduling_latency_s":1.002961}1531vm-test-run-nixbot-gitlab> gitlab # [  152.763794] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.167Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b030f1dc864379b0fda5db11","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b030f1dc864379b0fda5db11: done: 0.015792 sec","job_status":"done","queue_duration_s":1.002934,"scheduling_latency_s":1.002961,"redis_calls":13,"redis_duration_s":0.003096,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001032,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002064,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010009,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015792,"completed_at":"2026-06-29T10:45:38.167Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1532vm-test-run-nixbot-gitlab> gitlab # [  152.827231] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.169Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"61cfdef5c870638c847ce69d","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-61cfdef5c870638c847ce69d: start","job_status":"start","queue_duration_s":1.020215,"scheduling_latency_s":1.020236}1533vm-test-run-nixbot-gitlab> gitlab # [  152.898386] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.185Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"61cfdef5c870638c847ce69d","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-61cfdef5c870638c847ce69d: done: 0.015917 sec","job_status":"done","queue_duration_s":1.020215,"scheduling_latency_s":1.020236,"redis_calls":13,"redis_duration_s":0.0029230000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002023,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010086,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015917,"completed_at":"2026-06-29T10:45:38.185Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1534vm-test-run-nixbot-gitlab> gitlab # [  152.908207] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.186Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"46b60d46f72eb71aa4674222","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-46b60d46f72eb71aa4674222: start","job_status":"start","queue_duration_s":1.037474,"scheduling_latency_s":1.037497}1535vm-test-run-nixbot-gitlab> gitlab # [  152.992653] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.202Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"46b60d46f72eb71aa4674222","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-46b60d46f72eb71aa4674222: done: 0.015593 sec","job_status":"done","queue_duration_s":1.037474,"scheduling_latency_s":1.037497,"redis_calls":13,"redis_duration_s":0.003009,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000973,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002036,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009918,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015593,"completed_at":"2026-06-29T10:45:38.202Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1536vm-test-run-nixbot-gitlab> gitlab # [  152.994399] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.203Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8bd87ae41b657c38c1588d72","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8bd87ae41b657c38c1588d72: start","job_status":"start","queue_duration_s":1.054368,"scheduling_latency_s":1.054389}1537vm-test-run-nixbot-gitlab> gitlab # [  153.063223] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.218Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8bd87ae41b657c38c1588d72","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8bd87ae41b657c38c1588d72: done: 0.015429 sec","job_status":"done","queue_duration_s":1.054368,"scheduling_latency_s":1.054389,"redis_calls":13,"redis_duration_s":0.0029959999999999995,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000975,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002021,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009863,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015429,"completed_at":"2026-06-29T10:45:38.218Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1538vm-test-run-nixbot-gitlab> gitlab # [  153.070300] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.220Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b1e01096bb6c91692d1a2d04","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b1e01096bb6c91692d1a2d04: start","job_status":"start","queue_duration_s":1.071204,"scheduling_latency_s":1.071232}1539vm-test-run-nixbot-gitlab> gitlab # [  153.090103] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.236Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b1e01096bb6c91692d1a2d04","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b1e01096bb6c91692d1a2d04: done: 0.015787 sec","job_status":"done","queue_duration_s":1.071204,"scheduling_latency_s":1.071232,"redis_calls":13,"redis_duration_s":0.00326,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000976,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002284,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009689,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015787,"completed_at":"2026-06-29T10:45:38.235Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1540vm-test-run-nixbot-gitlab> gitlab # [  153.153275] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.237Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"865c0b3d26ad220e8c354663","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-865c0b3d26ad220e8c354663: start","job_status":"start","queue_duration_s":1.088294,"scheduling_latency_s":1.088315}1541vm-test-run-nixbot-gitlab> gitlab # [  153.173367] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.252Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"865c0b3d26ad220e8c354663","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-865c0b3d26ad220e8c354663: done: 0.015498 sec","job_status":"done","queue_duration_s":1.088294,"scheduling_latency_s":1.088315,"redis_calls":13,"redis_duration_s":0.002933,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000884,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002049,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009693,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015498,"completed_at":"2026-06-29T10:45:38.252Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1542vm-test-run-nixbot-gitlab> gitlab # [  153.282202] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.254Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b487e48d36a416e3486da323","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b487e48d36a416e3486da323: start","job_status":"start","queue_duration_s":1.105254,"scheduling_latency_s":1.105275}1543vm-test-run-nixbot-gitlab> gitlab # [  153.283311] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.269Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b487e48d36a416e3486da323","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b487e48d36a416e3486da323: done: 0.01562 sec","job_status":"done","queue_duration_s":1.105254,"scheduling_latency_s":1.105275,"redis_calls":13,"redis_duration_s":0.003199,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000971,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002228,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009813,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01562,"completed_at":"2026-06-29T10:45:38.269Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1544vm-test-run-nixbot-gitlab> gitlab # [  153.317106] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.271Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b7b835b6361cfc0cb97637b9","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b7b835b6361cfc0cb97637b9: start","job_status":"start","queue_duration_s":1.122305,"scheduling_latency_s":1.122326}1545vm-test-run-nixbot-gitlab> gitlab # [  153.390056] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.286Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b7b835b6361cfc0cb97637b9","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b7b835b6361cfc0cb97637b9: done: 0.015486 sec","job_status":"done","queue_duration_s":1.122305,"scheduling_latency_s":1.122326,"redis_calls":13,"redis_duration_s":0.0030549999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000988,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002067,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009839,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015486,"completed_at":"2026-06-29T10:45:38.286Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1546vm-test-run-nixbot-gitlab> gitlab # [  153.398166] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.288Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"57e8af7e9196943fe9f6106f","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-57e8af7e9196943fe9f6106f: start","job_status":"start","queue_duration_s":1.139388,"scheduling_latency_s":1.139409}1547vm-test-run-nixbot-gitlab> gitlab # [  153.473305] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.303Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"57e8af7e9196943fe9f6106f","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-57e8af7e9196943fe9f6106f: done: 0.015398 sec","job_status":"done","queue_duration_s":1.139388,"scheduling_latency_s":1.139409,"redis_calls":13,"redis_duration_s":0.003109,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001037,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002072,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009698,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015398,"completed_at":"2026-06-29T10:45:38.303Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1548vm-test-run-nixbot-gitlab> gitlab # [  153.478846] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.305Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d5cca5329a5c0695e7abbd18","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d5cca5329a5c0695e7abbd18: start","job_status":"start","queue_duration_s":1.156254,"scheduling_latency_s":1.156275}1549vm-test-run-nixbot-gitlab> gitlab # [  153.498943] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.320Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d5cca5329a5c0695e7abbd18","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d5cca5329a5c0695e7abbd18: done: 0.015291 sec","job_status":"done","queue_duration_s":1.156254,"scheduling_latency_s":1.156275,"redis_calls":13,"redis_duration_s":0.002887,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000941,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001946,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009819,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015291,"completed_at":"2026-06-29T10:45:38.320Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1550vm-test-run-nixbot-gitlab> gitlab # [  153.562198] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.322Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3e645549dafbedf6a03289c","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3e645549dafbedf6a03289c: start","job_status":"start","queue_duration_s":1.173066,"scheduling_latency_s":1.173087}1551vm-test-run-nixbot-gitlab> gitlab # [  153.647262] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.337Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3e645549dafbedf6a03289c","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3e645549dafbedf6a03289c: done: 0.015658 sec","job_status":"done","queue_duration_s":1.173066,"scheduling_latency_s":1.173087,"redis_calls":13,"redis_duration_s":0.0029189999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00098,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001939,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009876,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015658,"completed_at":"2026-06-29T10:45:38.337Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1552vm-test-run-nixbot-gitlab> gitlab # [  153.649109] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.339Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4213cd401d1d2c7c01383fac","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4213cd401d1d2c7c01383fac: start","job_status":"start","queue_duration_s":1.190279,"scheduling_latency_s":1.190334}1553vm-test-run-nixbot-gitlab> gitlab # [  153.663209] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.355Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4213cd401d1d2c7c01383fac","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4213cd401d1d2c7c01383fac: done: 0.015864 sec","job_status":"done","queue_duration_s":1.190279,"scheduling_latency_s":1.190334,"redis_calls":13,"redis_duration_s":0.002976,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00096,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002016,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010003,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015864,"completed_at":"2026-06-29T10:45:38.355Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1554vm-test-run-nixbot-gitlab> gitlab # [  153.779056] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.356Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8eb2c2b498dd53226e2396a3","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8eb2c2b498dd53226e2396a3: start","job_status":"start","queue_duration_s":1.207597,"scheduling_latency_s":1.207622}1555vm-test-run-nixbot-gitlab> gitlab # [  153.780233] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.372Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8eb2c2b498dd53226e2396a3","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8eb2c2b498dd53226e2396a3: done: 0.0154 sec","job_status":"done","queue_duration_s":1.207597,"scheduling_latency_s":1.207622,"redis_calls":13,"redis_duration_s":0.003136,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002081,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009901,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0154,"completed_at":"2026-06-29T10:45:38.371Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1556vm-test-run-nixbot-gitlab> gitlab # [  153.807086] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.373Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c5b2696204a7f9373a91e8d","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c5b2696204a7f9373a91e8d: start","job_status":"start","queue_duration_s":1.224289,"scheduling_latency_s":1.22431}1557vm-test-run-nixbot-gitlab> gitlab # [  153.827110] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.388Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c5b2696204a7f9373a91e8d","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c5b2696204a7f9373a91e8d: done: 0.015342 sec","job_status":"done","queue_duration_s":1.224289,"scheduling_latency_s":1.22431,"redis_calls":13,"redis_duration_s":0.0029029999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000958,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001945,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009748,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015342,"completed_at":"2026-06-29T10:45:38.388Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1558vm-test-run-nixbot-gitlab> gitlab # [  153.907930] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.390Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e43bd377b4000ad2d86d759","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e43bd377b4000ad2d86d759: start","job_status":"start","queue_duration_s":1.241053,"scheduling_latency_s":1.241074}1559vm-test-run-nixbot-gitlab> gitlab # [  153.909073] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.405Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e43bd377b4000ad2d86d759","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e43bd377b4000ad2d86d759: done: 0.015658 sec","job_status":"done","queue_duration_s":1.241053,"scheduling_latency_s":1.241074,"redis_calls":13,"redis_duration_s":0.0030020000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000985,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002017,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00977,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015658,"completed_at":"2026-06-29T10:45:38.405Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1560vm-test-run-nixbot-gitlab> gitlab # [  153.978359] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.407Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e95c39f5470bf19ae444eed9","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e95c39f5470bf19ae444eed9: start","job_status":"start","queue_duration_s":1.258092,"scheduling_latency_s":1.258112}1561vm-test-run-nixbot-gitlab> gitlab # [  154.049573] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.422Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e95c39f5470bf19ae444eed9","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e95c39f5470bf19ae444eed9: done: 0.015482 sec","job_status":"done","queue_duration_s":1.258092,"scheduling_latency_s":1.258112,"redis_calls":13,"redis_duration_s":0.0030350000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00092,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002115,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009618,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015482,"completed_at":"2026-06-29T10:45:38.422Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1562vm-test-run-nixbot-gitlab> gitlab # [  154.051258] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.424Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"de55acc07691b04a8a2a0317","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-de55acc07691b04a8a2a0317: start","job_status":"start","queue_duration_s":1.274985,"scheduling_latency_s":1.275005}1563vm-test-run-nixbot-gitlab> gitlab # [  154.070256] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.439Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"de55acc07691b04a8a2a0317","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-de55acc07691b04a8a2a0317: done: 0.015858 sec","job_status":"done","queue_duration_s":1.274985,"scheduling_latency_s":1.275005,"redis_calls":13,"redis_duration_s":0.002906,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000938,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001968,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009791,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015858,"completed_at":"2026-06-29T10:45:38.439Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1564vm-test-run-nixbot-gitlab> gitlab # [  154.144467] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.441Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7f0de02e26859a7d75be85da","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7f0de02e26859a7d75be85da: start","job_status":"start","queue_duration_s":1.292258,"scheduling_latency_s":1.29228}1565vm-test-run-nixbot-gitlab> gitlab # [  154.213053] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:45:38.456Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7f0de02e26859a7d75be85da","created_at":"2026-06-29T10:45:37.084Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-29a24b2440d1407b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a36aa6ea6fd462d8c130f6f0344b04da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:45:37.149Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7f0de02e26859a7d75be85da: done: 0.015304 sec","job_status":"done","queue_duration_s":1.292258,"scheduling_latency_s":1.29228,"redis_calls":13,"redis_duration_s":0.00302,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000983,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002037,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009647,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015304,"completed_at":"2026-06-29T10:45:38.456Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1566vm-test-run-nixbot-gitlab> gitlab # [  154.543709] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 4011007591 wd_nsec: 40110058961567vm-test-run-nixbot-gitlab> gitlab # [  169.725134] setup-gitlab-start[2700]: W, [2026-06-29T10:45:57.697929 #2700]  WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1568vm-test-run-nixbot-gitlab> gitlab # [  173.612114] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:01.584Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"a2b594654ccec2771946fef7","created_at":"2026-06-29T10:46:01.575Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6003efee5803779525b0b18132fd92f6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-29T10:46:01.580Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-a2b594654ccec2771946fef7: start","job_status":"start","queue_duration_s":0.003775,"scheduling_latency_s":0.003868}1569vm-test-run-nixbot-gitlab> gitlab # [  173.779469] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:01.752Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"a2b594654ccec2771946fef7","created_at":"2026-06-29T10:46:01.575Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6003efee5803779525b0b18132fd92f6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-29T10:46:01.580Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-a2b594654ccec2771946fef7: done: 0.168691 sec","job_status":"done","queue_duration_s":0.003775,"scheduling_latency_s":0.003868,"redis_calls":8,"redis_duration_s":0.023871000000000003,"redis_read_bytes":133,"redis_write_bytes":811,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000541,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":70,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.022336,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000994,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026078,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.168691,"completed_at":"2026-06-29T10:46:01.752Z","load_balancing_strategy":"primary","db_duration_s":0.0106,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1570vm-test-run-nixbot-gitlab> gitlab # [  173.866409] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:01.839Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"9e6e2a0c1cef18584ffd7d12","created_at":"2026-06-29T10:46:01.656Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c26daa97ab62db2af99ef48fcc4a4265","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:01.707Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-9e6e2a0c1cef18584ffd7d12: start","job_status":"start","queue_duration_s":0.131988,"scheduling_latency_s":0.132075}1571vm-test-run-nixbot-gitlab> gitlab # [  173.954457] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:01.927Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"9e6e2a0c1cef18584ffd7d12","created_at":"2026-06-29T10:46:01.656Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c26daa97ab62db2af99ef48fcc4a4265","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:01.707Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-9e6e2a0c1cef18584ffd7d12: done: 0.088231 sec","job_status":"done","queue_duration_s":0.131988,"scheduling_latency_s":0.132075,"redis_calls":7,"redis_duration_s":0.011363,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.007149,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004214,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025958,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.088231,"completed_at":"2026-06-29T10:46:01.927Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.023131,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1572vm-test-run-nixbot-gitlab> gitlab # [  174.027228] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:02.000Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"881b5c0a18b444638647c275","created_at":"2026-06-29T10:46:01.723Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f3663f4f927f34a296839010e6cab5b","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:01.728Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-881b5c0a18b444638647c275: start","job_status":"start","queue_duration_s":0.271512,"scheduling_latency_s":0.271576}1573vm-test-run-nixbot-gitlab> gitlab # [  174.136875] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:02.109Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"881b5c0a18b444638647c275","created_at":"2026-06-29T10:46:01.723Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f3663f4f927f34a296839010e6cab5b","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:01.728Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-881b5c0a18b444638647c275: done: 0.109366 sec","job_status":"done","queue_duration_s":0.271512,"scheduling_latency_s":0.271576,"redis_calls":10,"redis_duration_s":0.044674,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.007972,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.036702,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026916,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.008558909,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.109366,"completed_at":"2026-06-29T10:46:02.109Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006215,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1574vm-test-run-nixbot-gitlab> gitlab # [  174.229626] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:02.202Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"7259d4aedc52a56b77e3ecc5","created_at":"2026-06-29T10:46:01.736Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f03f45b3f52b6b49c3dc7c7ad64ad74","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:01.741Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-7259d4aedc52a56b77e3ecc5: start","job_status":"start","queue_duration_s":0.46083,"scheduling_latency_s":0.460893}1575vm-test-run-nixbot-gitlab> gitlab # [  174.322295] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:02.295Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"7259d4aedc52a56b77e3ecc5","created_at":"2026-06-29T10:46:01.736Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f03f45b3f52b6b49c3dc7c7ad64ad74","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:01.741Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-7259d4aedc52a56b77e3ecc5: done: 0.093207 sec","job_status":"done","queue_duration_s":0.46083,"scheduling_latency_s":0.460893,"redis_calls":10,"redis_duration_s":0.027833999999999998,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004125,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005737,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.017972,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028214,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000490845,"duration_s":0.093207,"completed_at":"2026-06-29T10:46:02.295Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024083,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1576vm-test-run-nixbot-gitlab> gitlab # [  174.394518] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:02.367Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"97925ae00a5a6e93f3672215","created_at":"2026-06-29T10:46:01.751Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a4f29edc1ad943674cf3fcb7c3890fb9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:01.837Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-97925ae00a5a6e93f3672215: start","job_status":"start","queue_duration_s":0.52997,"scheduling_latency_s":0.530036}1577vm-test-run-nixbot-gitlab> gitlab # [  174.477497] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:02.450Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"97925ae00a5a6e93f3672215","created_at":"2026-06-29T10:46:01.751Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a4f29edc1ad943674cf3fcb7c3890fb9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:01.837Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-97925ae00a5a6e93f3672215: done: 0.083137 sec","job_status":"done","queue_duration_s":0.52997,"scheduling_latency_s":0.530036,"redis_calls":9,"redis_duration_s":0.026812,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.018174,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.008638,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02371,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.003122464,"exclusive_lock_hold_duration_s":0.005051759,"duration_s":0.083137,"completed_at":"2026-06-29T10:46:02.450Z","load_balancing_strategy":"primary","db_duration_s":0.001866,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1578vm-test-run-nixbot-gitlab> gitlab # [  174.491961] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:02.465Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"86cb9332dabdc629d4cc62c3","created_at":"2026-06-29T10:46:01.861Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a0577430b32d7b110f05656cab8b933a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:01.869Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-86cb9332dabdc629d4cc62c3: start","job_status":"start","queue_duration_s":0.595786,"scheduling_latency_s":0.595844}1579vm-test-run-nixbot-gitlab> gitlab # [  174.552217] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:02.525Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"86cb9332dabdc629d4cc62c3","created_at":"2026-06-29T10:46:01.861Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a0577430b32d7b110f05656cab8b933a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:01.869Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-86cb9332dabdc629d4cc62c3: done: 0.060098 sec","job_status":"done","queue_duration_s":0.595786,"scheduling_latency_s":0.595844,"redis_calls":7,"redis_duration_s":0.014584,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.008783,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.005801,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017088,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.060098,"completed_at":"2026-06-29T10:46:02.525Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1580vm-test-run-nixbot-gitlab> gitlab # [  174.564594] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:02.529Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"bb60d864d2f67c823dfe37c4","created_at":"2026-06-29T10:46:01.889Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ea0ffe4441f21417bac8d62ed084d58e","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:01.909Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-bb60d864d2f67c823dfe37c4: start","job_status":"start","queue_duration_s":0.619442,"scheduling_latency_s":0.619511}1581vm-test-run-nixbot-gitlab> gitlab # [  174.671634] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:02.644Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"bb60d864d2f67c823dfe37c4","created_at":"2026-06-29T10:46:01.889Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ea0ffe4441f21417bac8d62ed084d58e","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:01.909Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-bb60d864d2f67c823dfe37c4: done: 0.115038 sec","job_status":"done","queue_duration_s":0.619442,"scheduling_latency_s":0.619511,"redis_calls":12,"redis_duration_s":0.005011,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000703,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002519,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001789,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.076,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.076,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017835,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000690311,"exclusive_lock_hold_duration_s":0.0040357069999999995,"duration_s":0.115038,"completed_at":"2026-06-29T10:46:02.644Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.076771,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1582vm-test-run-nixbot-gitlab> gitlab # [  174.674640] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:02.645Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"02b0f971cbcf668ad02b4131","created_at":"2026-06-29T10:46:01.921Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0f39965e97dee237bc141706d2790feb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:01.925Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-02b0f971cbcf668ad02b4131: start","job_status":"start","queue_duration_s":0.720588,"scheduling_latency_s":0.72061}1583vm-test-run-nixbot-gitlab> gitlab # [  174.851559] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:02.824Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"02b0f971cbcf668ad02b4131","created_at":"2026-06-29T10:46:01.921Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0f39965e97dee237bc141706d2790feb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:01.925Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-02b0f971cbcf668ad02b4131: done: 0.178625 sec","job_status":"done","queue_duration_s":0.720588,"scheduling_latency_s":0.72061,"redis_calls":9,"redis_duration_s":0.004883,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002506,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002377,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.022,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041688,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001058235,"exclusive_lock_hold_duration_s":0.057652859,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.178625,"completed_at":"2026-06-29T10:46:02.824Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.045625,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1584vm-test-run-nixbot-gitlab> gitlab # [  174.856527] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:02.828Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"0a2aedc1b67a38c12a4ff228","created_at":"2026-06-29T10:46:02.063Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"02e55b0177ab7d7fdb36df2ef41538f8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:02.077Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-0a2aedc1b67a38c12a4ff228: start","job_status":"start","queue_duration_s":0.750643,"scheduling_latency_s":0.750693}1585vm-test-run-nixbot-gitlab> gitlab # [  174.897835] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:02.847Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"0a2aedc1b67a38c12a4ff228","created_at":"2026-06-29T10:46:02.063Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"02e55b0177ab7d7fdb36df2ef41538f8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:02.077Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-0a2aedc1b67a38c12a4ff228: done: 0.019453 sec","job_status":"done","queue_duration_s":0.750643,"scheduling_latency_s":0.750693,"redis_calls":7,"redis_duration_s":0.0020280000000000003,"redis_read_bytes":5,"redis_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001381,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000647,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010123,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019453,"completed_at":"2026-06-29T10:46:02.847Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1586vm-test-run-nixbot-gitlab> gitlab # [  175.010259] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:02.849Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"171de99ba77d6073b4d95da0","created_at":"2026-06-29T10:46:02.094Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0afaa7038505c40662f6764761125593","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:02.100Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-171de99ba77d6073b4d95da0: start","job_status":"start","queue_duration_s":0.748996,"scheduling_latency_s":0.749018}1587vm-test-run-nixbot-gitlab> gitlab # [  175.100626] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:03.073Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"171de99ba77d6073b4d95da0","created_at":"2026-06-29T10:46:02.094Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0afaa7038505c40662f6764761125593","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:02.100Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-171de99ba77d6073b4d95da0: done: 0.224285 sec","job_status":"done","queue_duration_s":0.748996,"scheduling_latency_s":0.749018,"redis_calls":9,"redis_duration_s":0.004459,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001886,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002573,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028953,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001637917,"exclusive_lock_hold_duration_s":0.022008105,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.224285,"completed_at":"2026-06-29T10:46:03.073Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.188064,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1588vm-test-run-nixbot-gitlab> gitlab # [  175.173229] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:03.146Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"2eb4971c4b26e823597184e5","created_at":"2026-06-29T10:46:02.195Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d85fdbba74a99120adbfc4d133d2ffa5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:02.201Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-2eb4971c4b26e823597184e5: start","job_status":"start","queue_duration_s":0.944518,"scheduling_latency_s":0.944584}1589vm-test-run-nixbot-gitlab> gitlab # [  175.193518] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:03.167Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1590vm-test-run-nixbot-gitlab> gitlab # [  175.205557] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:03.178Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"2eb4971c4b26e823597184e5","created_at":"2026-06-29T10:46:02.195Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d85fdbba74a99120adbfc4d133d2ffa5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:02.201Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-2eb4971c4b26e823597184e5: done: 0.032424 sec","job_status":"done","queue_duration_s":0.944518,"scheduling_latency_s":0.944584,"redis_calls":7,"redis_duration_s":0.004695,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003301,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001394,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018574,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032424,"completed_at":"2026-06-29T10:46:03.178Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1591vm-test-run-nixbot-gitlab> gitlab # [  175.209203] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:03.182Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"d0f23ef86499c9bd530a7d78","created_at":"2026-06-29T10:46:02.236Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dd3c20b45f42cac2309e0dbc9015c952","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:02.246Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-d0f23ef86499c9bd530a7d78: start","job_status":"start","queue_duration_s":0.935812,"scheduling_latency_s":0.935863}1592vm-test-run-nixbot-gitlab> gitlab # [  175.238731] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:03.211Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"d0f23ef86499c9bd530a7d78","created_at":"2026-06-29T10:46:02.236Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dd3c20b45f42cac2309e0dbc9015c952","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:02.246Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-d0f23ef86499c9bd530a7d78: done: 0.029561 sec","job_status":"done","queue_duration_s":0.935812,"scheduling_latency_s":0.935863,"redis_calls":9,"redis_duration_s":0.00489,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002609,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002281,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01577,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00101214,"exclusive_lock_hold_duration_s":0.00356414,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.029561,"completed_at":"2026-06-29T10:46:03.211Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001801,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1593vm-test-run-nixbot-gitlab> gitlab # [  175.291576] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:03.214Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"a8ff317215705f1db29f35c8","created_at":"2026-06-29T10:46:02.269Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"05d1e541936972dd2a55c7ff112ce7af","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:02.273Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-a8ff317215705f1db29f35c8: start","job_status":"start","queue_duration_s":0.940627,"scheduling_latency_s":0.940647}1594vm-test-run-nixbot-gitlab> gitlab # [  175.315279] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:03.221Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1595vm-test-run-nixbot-gitlab> gitlab # [  175.320120] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:03.225Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"a8ff317215705f1db29f35c8","created_at":"2026-06-29T10:46:02.269Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"05d1e541936972dd2a55c7ff112ce7af","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:02.273Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-a8ff317215705f1db29f35c8: done: 0.011636 sec","job_status":"done","queue_duration_s":0.940627,"scheduling_latency_s":0.940647,"redis_calls":7,"redis_duration_s":0.001686,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001216,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00047,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006984,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011636,"completed_at":"2026-06-29T10:46:03.225Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1596vm-test-run-nixbot-gitlab> gitlab # [  175.424891] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:03.227Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"96251789e1de4098eb94ac8c","created_at":"2026-06-29T10:46:02.285Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4fc14921c551cf74197f18123c44d706","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:02.288Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-96251789e1de4098eb94ac8c: start","job_status":"start","queue_duration_s":0.938086,"scheduling_latency_s":0.938106}1597vm-test-run-nixbot-gitlab> gitlab # [  175.679968] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:03.652Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"96251789e1de4098eb94ac8c","created_at":"2026-06-29T10:46:02.285Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4fc14921c551cf74197f18123c44d706","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:02.288Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-96251789e1de4098eb94ac8c: done: 0.425921 sec","job_status":"done","queue_duration_s":0.938086,"scheduling_latency_s":0.938106,"redis_calls":571,"redis_duration_s":0.14981999999999998,"redis_read_bytes":569,"redis_write_bytes":49487,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001182,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.148638,"redis_shared_state_read_bytes":565,"redis_shared_state_write_bytes":49017,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.312726,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.425921,"completed_at":"2026-06-29T10:46:03.652Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1598vm-test-run-nixbot-gitlab> gitlab # [  175.783775] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:03.757Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"b4079e94d9dca15861ca8b9e","created_at":"2026-06-29T10:46:02.296Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0bf71a7d98e2466ba85fa6707d8df8da","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:02.365Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-b4079e94d9dca15861ca8b9e: start","job_status":"start","queue_duration_s":1.391383,"scheduling_latency_s":1.39155}1599vm-test-run-nixbot-gitlab> gitlab # [  175.822586] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:03.796Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1600vm-test-run-nixbot-gitlab> gitlab # [  175.834705] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:03.807Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"b4079e94d9dca15861ca8b9e","created_at":"2026-06-29T10:46:02.296Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0bf71a7d98e2466ba85fa6707d8df8da","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:02.365Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-b4079e94d9dca15861ca8b9e: done: 0.050883 sec","job_status":"done","queue_duration_s":1.391383,"scheduling_latency_s":1.39155,"redis_calls":7,"redis_duration_s":0.004033,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002945,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001088,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018489,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.050883,"completed_at":"2026-06-29T10:46:03.807Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1601vm-test-run-nixbot-gitlab> gitlab # [  175.902164] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:03.875Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"1f9742e71ec09f042131a0c6","created_at":"2026-06-29T10:46:02.419Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3fbfa3324c205a57f4a107c0c37b7354","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:02.441Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-1f9742e71ec09f042131a0c6: start","job_status":"start","queue_duration_s":1.433957,"scheduling_latency_s":1.43402}1602vm-test-run-nixbot-gitlab> gitlab # [  175.964452] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:03.937Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"1f9742e71ec09f042131a0c6","created_at":"2026-06-29T10:46:02.419Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3fbfa3324c205a57f4a107c0c37b7354","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:02.441Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-1f9742e71ec09f042131a0c6: done: 0.062335 sec","job_status":"done","queue_duration_s":1.433957,"scheduling_latency_s":1.43402,"redis_calls":12,"redis_duration_s":0.007417,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.001318,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001129,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003931,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001039,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025329,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000920508,"duration_s":0.062335,"completed_at":"2026-06-29T10:46:03.937Z","load_balancing_strategy":"primary","db_duration_s":0.001157,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1603vm-test-run-nixbot-gitlab> gitlab # [  176.034727] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:04.007Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::PipelineArtifacts::ExpireArtifactsWorker","jid":"eae9121ed7bf870bbcf03a43","created_at":"2026-06-29T10:46:02.457Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9ed97cdd4aec90a257cb0067b8238849","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:35dc78c13bdf78a98c09fc2795030a4961be87ad882288b77a6a3c9be677e4d5","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:02.477Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::ExpireArtifactsWorker JID-eae9121ed7bf870bbcf03a43: start","job_status":"start","queue_duration_s":1.530628,"scheduling_latency_s":1.530691}1604vm-test-run-nixbot-gitlab> gitlab # [  176.091507] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:04.064Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::PipelineArtifacts::ExpireArtifactsWorker","jid":"eae9121ed7bf870bbcf03a43","created_at":"2026-06-29T10:46:02.457Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9ed97cdd4aec90a257cb0067b8238849","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:35dc78c13bdf78a98c09fc2795030a4961be87ad882288b77a6a3c9be677e4d5","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:02.477Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::ExpireArtifactsWorker JID-eae9121ed7bf870bbcf03a43: done: 0.056912 sec","job_status":"done","queue_duration_s":1.530628,"scheduling_latency_s":1.530691,"redis_calls":12,"redis_duration_s":0.007546000000000001,"redis_read_bytes":11,"redis_write_bytes":1481,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001216,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003718,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002612,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":548,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031235,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00121887,"exclusive_lock_hold_duration_s":0.014666669,"extra.ci_pipeline_artifacts_expire_artifacts_worker.destroyed_pipeline_artifacts_count":0,"duration_s":0.056912,"completed_at":"2026-06-29T10:46:04.064Z","load_balancing_strategy":"primary","db_duration_s":0.007502,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1605vm-test-run-nixbot-gitlab> gitlab # [  176.163145] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:04.136Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"3335052d537c41c746f58e1e","created_at":"2026-06-29T10:46:02.498Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8bf430cf28c299479149283082a3d4ba","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:02.509Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-3335052d537c41c746f58e1e: start","job_status":"start","queue_duration_s":1.626479,"scheduling_latency_s":1.626544}1606vm-test-run-nixbot-gitlab> gitlab # [  176.401262] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:04.374Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"3335052d537c41c746f58e1e","created_at":"2026-06-29T10:46:02.498Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8bf430cf28c299479149283082a3d4ba","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:02.509Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-3335052d537c41c746f58e1e: done: 0.238659 sec","job_status":"done","queue_duration_s":1.626479,"scheduling_latency_s":1.626544,"redis_calls":11,"redis_duration_s":0.006661,"redis_read_bytes":9,"redis_write_bytes":1262,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001287,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":275,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002871,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002503,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":124,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":124,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.053,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":124,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":80,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.053,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.152209,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001217194,"exclusive_lock_hold_duration_s":0.203402946,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.238659,"completed_at":"2026-06-29T10:46:04.374Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.087763,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1607vm-test-run-nixbot-gitlab> gitlab # [  176.406377] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:04.376Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"4233419aede416f9a8ace80d","created_at":"2026-06-29T10:46:02.531Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"80e7eebd779fa82f8f36cc2621f9d0ea","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:02.537Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-4233419aede416f9a8ace80d: start","job_status":"start","queue_duration_s":1.838399,"scheduling_latency_s":1.83842}1608vm-test-run-nixbot-gitlab> gitlab # [  176.452388] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:04.383Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1609vm-test-run-nixbot-gitlab> gitlab # [  176.458238] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:04.387Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"4233419aede416f9a8ace80d","created_at":"2026-06-29T10:46:02.531Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"80e7eebd779fa82f8f36cc2621f9d0ea","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:02.537Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-4233419aede416f9a8ace80d: done: 0.011728 sec","job_status":"done","queue_duration_s":1.838399,"scheduling_latency_s":1.83842,"redis_calls":7,"redis_duration_s":0.001651,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001215,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000436,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006956,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011728,"completed_at":"2026-06-29T10:46:04.387Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1610vm-test-run-nixbot-gitlab> gitlab # [  176.534403] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:04.389Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"c8624ae059696ab746ceab4b","created_at":"2026-06-29T10:46:02.584Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c1d7dad36847f1f9cc8f3559cd6d3df0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:02.585Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-c8624ae059696ab746ceab4b: start","job_status":"start","queue_duration_s":1.803197,"scheduling_latency_s":1.803217}1611vm-test-run-nixbot-gitlab> gitlab # [  176.631665] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:04.605Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"c8624ae059696ab746ceab4b","created_at":"2026-06-29T10:46:02.584Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c1d7dad36847f1f9cc8f3559cd6d3df0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:02.585Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-c8624ae059696ab746ceab4b: done: 0.21589 sec","job_status":"done","queue_duration_s":1.803197,"scheduling_latency_s":1.803217,"redis_calls":14,"redis_duration_s":0.0041210000000000005,"redis_read_bytes":15,"redis_write_bytes":4030,"redis_queues_calls":2,"redis_queues_duration_s":0.000457,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002683,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000981,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.191,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.191,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018454,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.21589,"completed_at":"2026-06-29T10:46:04.605Z","load_balancing_strategy":"primary","db_duration_s":0.191109,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1612vm-test-run-nixbot-gitlab> gitlab # [  176.699087] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:04.672Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"4ec0cba0547f3d66e6fb6b34","created_at":"2026-06-29T10:46:02.588Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:02.589Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-4ec0cba0547f3d66e6fb6b34: start","job_status":"start","queue_duration_s":2.081876,"scheduling_latency_s":2.082021}1613vm-test-run-nixbot-gitlab> gitlab # [  176.798568] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:04.771Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"4ec0cba0547f3d66e6fb6b34","created_at":"2026-06-29T10:46:02.588Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:02.589Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-4ec0cba0547f3d66e6fb6b34: done: 0.100162 sec","job_status":"done","queue_duration_s":2.081876,"scheduling_latency_s":2.082021,"redis_calls":60,"redis_duration_s":0.024402,"redis_read_bytes":108,"redis_write_bytes":41059,"redis_queues_calls":2,"redis_queues_duration_s":0.001301,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.022074,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001027,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.073466,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.100162,"completed_at":"2026-06-29T10:46:04.771Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1614vm-test-run-nixbot-gitlab> gitlab # [  176.892107] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:04.865Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"7cbaf1acf1e3ab840e81103c","created_at":"2026-06-29T10:46:02.592Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"efc2d0f5d183ab9f72fd8ff68a495416","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-29T10:46:02.594Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-7cbaf1acf1e3ab840e81103c: start","job_status":"start","queue_duration_s":2.270967,"scheduling_latency_s":2.271029}1615vm-test-run-nixbot-gitlab> gitlab # [  176.912454] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:04.885Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1616vm-test-run-nixbot-gitlab> gitlab # [  176.924571] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:04.897Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"7cbaf1acf1e3ab840e81103c","created_at":"2026-06-29T10:46:02.592Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"efc2d0f5d183ab9f72fd8ff68a495416","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-29T10:46:02.594Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-7cbaf1acf1e3ab840e81103c: done: 0.032583 sec","job_status":"done","queue_duration_s":2.270967,"scheduling_latency_s":2.271029,"redis_calls":7,"redis_duration_s":0.004708,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003319,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001389,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019316,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032583,"completed_at":"2026-06-29T10:46:04.897Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1617vm-test-run-nixbot-gitlab> gitlab # [  176.990203] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:04.963Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"65207f11cfe17048fe8d6696","created_at":"2026-06-29T10:46:02.597Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7aa34889ef8112366765c9422e4f875c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-29T10:46:02.598Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-65207f11cfe17048fe8d6696: start","job_status":"start","queue_duration_s":2.364172,"scheduling_latency_s":2.364235}1618vm-test-run-nixbot-gitlab> gitlab # [  177.027604] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.001Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1619vm-test-run-nixbot-gitlab> gitlab # [  177.042957] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.016Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"65207f11cfe17048fe8d6696","created_at":"2026-06-29T10:46:02.597Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7aa34889ef8112366765c9422e4f875c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-29T10:46:02.598Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-65207f11cfe17048fe8d6696: done: 0.053208 sec","job_status":"done","queue_duration_s":2.364172,"scheduling_latency_s":2.364235,"redis_calls":7,"redis_duration_s":0.007014,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006169,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000845,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018454,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.053208,"completed_at":"2026-06-29T10:46:05.015Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1620vm-test-run-nixbot-gitlab> gitlab # [  177.109437] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.082Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"9746ec400f65b601541f3a12","created_at":"2026-06-29T10:46:02.604Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"63b5d22335583ce295e53d7c58f89537","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:02.605Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-9746ec400f65b601541f3a12: start","job_status":"start","queue_duration_s":2.476949,"scheduling_latency_s":2.477012}1621vm-test-run-nixbot-gitlab> gitlab # [  177.142105] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.115Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"9746ec400f65b601541f3a12","created_at":"2026-06-29T10:46:02.604Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"63b5d22335583ce295e53d7c58f89537","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:02.605Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-9746ec400f65b601541f3a12: done: 0.032588 sec","job_status":"done","queue_duration_s":2.476949,"scheduling_latency_s":2.477012,"redis_calls":7,"redis_duration_s":0.004491,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003123,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001368,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018138,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032588,"completed_at":"2026-06-29T10:46:05.114Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001808,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1622vm-test-run-nixbot-gitlab> gitlab # [  177.145244] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.118Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"81177a4e86c0681710ddae38","created_at":"2026-06-29T10:46:02.609Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7fd9ee0e29d09c1a409259f3cea2d6cb","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:02.610Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-81177a4e86c0681710ddae38: start","job_status":"start","queue_duration_s":2.508314,"scheduling_latency_s":2.508363}1623vm-test-run-nixbot-gitlab> gitlab # [  177.181917] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.155Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"81177a4e86c0681710ddae38","created_at":"2026-06-29T10:46:02.609Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-4e3b12f4ea38408b","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7fd9ee0e29d09c1a409259f3cea2d6cb","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:02.610Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-81177a4e86c0681710ddae38: done: 0.036654 sec","job_status":"done","queue_duration_s":2.508314,"scheduling_latency_s":2.508363,"redis_calls":7,"redis_duration_s":0.00365,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002713,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000937,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015228,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.036654,"completed_at":"2026-06-29T10:46:05.155Z","load_balancing_strategy":"primary","db_duration_s":0.01028,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1624vm-test-run-nixbot-gitlab> gitlab # [  177.275457] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.157Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9fa5f67c137270ce0b4be450","created_at":"2026-06-29T10:46:04.589Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-655b5f4baa0142a2","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c1d7dad36847f1f9cc8f3559cd6d3df0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.597Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9fa5f67c137270ce0b4be450: start","job_status":"start","queue_duration_s":0.560669,"scheduling_latency_s":0.56071}1625vm-test-run-nixbot-gitlab> gitlab # [  177.277100] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.172Z","message":"No existing merge request to be cleaned up."}1626vm-test-run-nixbot-gitlab> gitlab # [  177.278618] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.179Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9fa5f67c137270ce0b4be450","created_at":"2026-06-29T10:46:04.589Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-655b5f4baa0142a2","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c1d7dad36847f1f9cc8f3559cd6d3df0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.597Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9fa5f67c137270ce0b4be450: done: 0.021551 sec","job_status":"done","queue_duration_s":0.560669,"scheduling_latency_s":0.56071,"redis_calls":10,"redis_duration_s":0.002381,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000934,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001447,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011012,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021551,"completed_at":"2026-06-29T10:46:05.179Z","load_balancing_strategy":"primary","db_duration_s":0.004017,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1627vm-test-run-nixbot-gitlab> gitlab # [  177.356131] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.180Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c2307a7f7a2f1545963c5ddb","created_at":"2026-06-29T10:46:04.589Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-655b5f4baa0142a2","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c1d7dad36847f1f9cc8f3559cd6d3df0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.597Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c2307a7f7a2f1545963c5ddb: start","job_status":"start","queue_duration_s":0.583485,"scheduling_latency_s":0.583505}1628vm-test-run-nixbot-gitlab> gitlab # [  177.357645] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.192Z","message":"No existing merge request to be cleaned up."}1629vm-test-run-nixbot-gitlab> gitlab # [  177.359406] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.198Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c2307a7f7a2f1545963c5ddb","created_at":"2026-06-29T10:46:04.589Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-655b5f4baa0142a2","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c1d7dad36847f1f9cc8f3559cd6d3df0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.597Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c2307a7f7a2f1545963c5ddb: done: 0.018444 sec","job_status":"done","queue_duration_s":0.583485,"scheduling_latency_s":0.583505,"redis_calls":10,"redis_duration_s":0.002656,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001163,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001493,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011246,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018444,"completed_at":"2026-06-29T10:46:05.198Z","load_balancing_strategy":"primary","db_duration_s":0.001883,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1630vm-test-run-nixbot-gitlab> gitlab # [  177.389222] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.200Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ff1b3054dc7a79e18df991ae","created_at":"2026-06-29T10:46:04.589Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-655b5f4baa0142a2","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c1d7dad36847f1f9cc8f3559cd6d3df0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.597Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ff1b3054dc7a79e18df991ae: start","job_status":"start","queue_duration_s":0.603271,"scheduling_latency_s":0.603293}1631vm-test-run-nixbot-gitlab> gitlab # [  177.409143] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.209Z","message":"No existing merge request to be cleaned up."}1632vm-test-run-nixbot-gitlab> gitlab # [  177.411989] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.215Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ff1b3054dc7a79e18df991ae","created_at":"2026-06-29T10:46:04.589Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-655b5f4baa0142a2","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c1d7dad36847f1f9cc8f3559cd6d3df0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.597Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ff1b3054dc7a79e18df991ae: done: 0.015033 sec","job_status":"done","queue_duration_s":0.603271,"scheduling_latency_s":0.603293,"redis_calls":10,"redis_duration_s":0.0023020000000000002,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000959,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001343,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009502,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015033,"completed_at":"2026-06-29T10:46:05.215Z","load_balancing_strategy":"primary","db_duration_s":0.001513,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1633vm-test-run-nixbot-gitlab> gitlab # [  177.501303] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.217Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7c2906af4bd5536136825da0","created_at":"2026-06-29T10:46:04.589Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-655b5f4baa0142a2","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c1d7dad36847f1f9cc8f3559cd6d3df0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.597Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7c2906af4bd5536136825da0: start","job_status":"start","queue_duration_s":0.619925,"scheduling_latency_s":0.619945}1634vm-test-run-nixbot-gitlab> gitlab # [  177.502382] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.227Z","message":"No existing merge request to be cleaned up."}1635vm-test-run-nixbot-gitlab> gitlab # [  177.503241] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.233Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7c2906af4bd5536136825da0","created_at":"2026-06-29T10:46:04.589Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-655b5f4baa0142a2","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c1d7dad36847f1f9cc8f3559cd6d3df0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.597Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7c2906af4bd5536136825da0: done: 0.016772 sec","job_status":"done","queue_duration_s":0.619925,"scheduling_latency_s":0.619945,"redis_calls":10,"redis_duration_s":0.003549,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001471,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002078,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009492,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016772,"completed_at":"2026-06-29T10:46:05.233Z","load_balancing_strategy":"primary","db_duration_s":0.002163,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1636vm-test-run-nixbot-gitlab> gitlab # [  177.578111] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.235Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b902b9f9a631c520f6e62766","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b902b9f9a631c520f6e62766: start","job_status":"start","queue_duration_s":0.470875,"scheduling_latency_s":0.470895}1637vm-test-run-nixbot-gitlab> gitlab # [  177.579306] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.251Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b902b9f9a631c520f6e62766","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b902b9f9a631c520f6e62766: done: 0.016168 sec","job_status":"done","queue_duration_s":0.470875,"scheduling_latency_s":0.470895,"redis_calls":13,"redis_duration_s":0.00425,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001197,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003053,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009686,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016168,"completed_at":"2026-06-29T10:46:05.251Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1638vm-test-run-nixbot-gitlab> gitlab # [  177.637909] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.252Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d07905da470c104ae47f0f3","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d07905da470c104ae47f0f3: start","job_status":"start","queue_duration_s":0.48837,"scheduling_latency_s":0.48839}1639vm-test-run-nixbot-gitlab> gitlab # [  177.657861] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.267Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d07905da470c104ae47f0f3","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d07905da470c104ae47f0f3: done: 0.014855 sec","job_status":"done","queue_duration_s":0.48837,"scheduling_latency_s":0.48839,"redis_calls":13,"redis_duration_s":0.003001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000929,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002072,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009394,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014855,"completed_at":"2026-06-29T10:46:05.267Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1640vm-test-run-nixbot-gitlab> gitlab # [  177.721437] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.269Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"50fd2dace84b3dcc72cc3e81","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-50fd2dace84b3dcc72cc3e81: start","job_status":"start","queue_duration_s":0.504526,"scheduling_latency_s":0.504546}1641vm-test-run-nixbot-gitlab> gitlab # [  177.790189] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.283Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"50fd2dace84b3dcc72cc3e81","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-50fd2dace84b3dcc72cc3e81: done: 0.014591 sec","job_status":"done","queue_duration_s":0.504526,"scheduling_latency_s":0.504546,"redis_calls":13,"redis_duration_s":0.002902,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000903,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001999,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00968,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014591,"completed_at":"2026-06-29T10:46:05.283Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1642vm-test-run-nixbot-gitlab> gitlab # [  177.801971] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.284Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f792712ba021e040a29f2e48","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f792712ba021e040a29f2e48: start","job_status":"start","queue_duration_s":0.520367,"scheduling_latency_s":0.520389}1643vm-test-run-nixbot-gitlab> gitlab # [  177.883517] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.300Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f792712ba021e040a29f2e48","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f792712ba021e040a29f2e48: done: 0.016008 sec","job_status":"done","queue_duration_s":0.520367,"scheduling_latency_s":0.520389,"redis_calls":13,"redis_duration_s":0.004313,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001275,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003038,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009739,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016008,"completed_at":"2026-06-29T10:46:05.300Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1644vm-test-run-nixbot-gitlab> gitlab # [  177.888744] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.302Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cbfc8cf858532dc02dc6d94b","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cbfc8cf858532dc02dc6d94b: start","job_status":"start","queue_duration_s":0.538073,"scheduling_latency_s":0.538094}1645vm-test-run-nixbot-gitlab> gitlab # [  177.902098] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.319Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cbfc8cf858532dc02dc6d94b","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cbfc8cf858532dc02dc6d94b: done: 0.016491 sec","job_status":"done","queue_duration_s":0.538073,"scheduling_latency_s":0.538094,"redis_calls":13,"redis_duration_s":0.004589,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001406,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003183,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00973,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016491,"completed_at":"2026-06-29T10:46:05.318Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1646vm-test-run-nixbot-gitlab> gitlab # [  177.965248] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.320Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d94381f1b8833838f24f5750","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d94381f1b8833838f24f5750: start","job_status":"start","queue_duration_s":0.55622,"scheduling_latency_s":0.556241}1647vm-test-run-nixbot-gitlab> gitlab # [  178.048111] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.347Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d94381f1b8833838f24f5750","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d94381f1b8833838f24f5750: done: 0.02702 sec","job_status":"done","queue_duration_s":0.55622,"scheduling_latency_s":0.556241,"redis_calls":13,"redis_duration_s":0.009787,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001653,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.008134,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013989,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02702,"completed_at":"2026-06-29T10:46:05.347Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1648vm-test-run-nixbot-gitlab> gitlab # [  178.050364] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.349Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"882cbf61a4d0193a2718abfd","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-882cbf61a4d0193a2718abfd: start","job_status":"start","queue_duration_s":0.585164,"scheduling_latency_s":0.585191}1649vm-test-run-nixbot-gitlab> gitlab # [  178.065914] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.364Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"882cbf61a4d0193a2718abfd","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-882cbf61a4d0193a2718abfd: done: 0.015268 sec","job_status":"done","queue_duration_s":0.585164,"scheduling_latency_s":0.585191,"redis_calls":13,"redis_duration_s":0.003025,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000925,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0021,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009552,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015268,"completed_at":"2026-06-29T10:46:05.364Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1650vm-test-run-nixbot-gitlab> gitlab # [  178.129082] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.366Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9c68168f1afcfbd96fd404e","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9c68168f1afcfbd96fd404e: start","job_status":"start","queue_duration_s":0.601722,"scheduling_latency_s":0.601744}1651vm-test-run-nixbot-gitlab> gitlab # [  178.148897] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.380Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9c68168f1afcfbd96fd404e","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9c68168f1afcfbd96fd404e: done: 0.01471 sec","job_status":"done","queue_duration_s":0.601722,"scheduling_latency_s":0.601744,"redis_calls":13,"redis_duration_s":0.003047,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000927,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00212,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009328,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01471,"completed_at":"2026-06-29T10:46:05.380Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1652vm-test-run-nixbot-gitlab> gitlab # [  178.222907] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.382Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"abcfc23fc52bd1a187340c8d","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-abcfc23fc52bd1a187340c8d: start","job_status":"start","queue_duration_s":0.61772,"scheduling_latency_s":0.61774}1653vm-test-run-nixbot-gitlab> gitlab # [  178.229172] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.397Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"abcfc23fc52bd1a187340c8d","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-abcfc23fc52bd1a187340c8d: done: 0.014903 sec","job_status":"done","queue_duration_s":0.61772,"scheduling_latency_s":0.61774,"redis_calls":13,"redis_duration_s":0.002983,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000882,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002101,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009205,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014903,"completed_at":"2026-06-29T10:46:05.397Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1654vm-test-run-nixbot-gitlab> gitlab # [  178.298538] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.398Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4298c4484e527988e8a52576","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4298c4484e527988e8a52576: start","job_status":"start","queue_duration_s":0.633897,"scheduling_latency_s":0.633917}1655vm-test-run-nixbot-gitlab> gitlab # [  178.310592] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.412Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4298c4484e527988e8a52576","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4298c4484e527988e8a52576: done: 0.014502 sec","job_status":"done","queue_duration_s":0.633897,"scheduling_latency_s":0.633917,"redis_calls":13,"redis_duration_s":0.002941,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000969,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001972,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009166,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014502,"completed_at":"2026-06-29T10:46:05.412Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1656vm-test-run-nixbot-gitlab> gitlab # [  178.393503] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.414Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"614f3c044dd4dc535f774dae","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-614f3c044dd4dc535f774dae: start","job_status":"start","queue_duration_s":0.649692,"scheduling_latency_s":0.649713}1657vm-test-run-nixbot-gitlab> gitlab # [  178.395313] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.428Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"614f3c044dd4dc535f774dae","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-614f3c044dd4dc535f774dae: done: 0.014584 sec","job_status":"done","queue_duration_s":0.649692,"scheduling_latency_s":0.649713,"redis_calls":13,"redis_duration_s":0.003055,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000918,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002137,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00926,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014584,"completed_at":"2026-06-29T10:46:05.428Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1658vm-test-run-nixbot-gitlab> gitlab # [  178.512409] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.430Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"232dcbcbbd6ba94f2bbfa506","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-232dcbcbbd6ba94f2bbfa506: start","job_status":"start","queue_duration_s":0.665592,"scheduling_latency_s":0.665613}1659vm-test-run-nixbot-gitlab> gitlab # [  178.514319] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.444Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"232dcbcbbd6ba94f2bbfa506","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-232dcbcbbd6ba94f2bbfa506: done: 0.014629 sec","job_status":"done","queue_duration_s":0.665592,"scheduling_latency_s":0.665613,"redis_calls":13,"redis_duration_s":0.003067,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000911,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002156,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009203,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014629,"completed_at":"2026-06-29T10:46:05.444Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1660vm-test-run-nixbot-gitlab> gitlab # [  178.540559] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.446Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5df76118a1086433997d3beb","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5df76118a1086433997d3beb: start","job_status":"start","queue_duration_s":0.681691,"scheduling_latency_s":0.681712}1661vm-test-run-nixbot-gitlab> gitlab # [  178.560478] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.460Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5df76118a1086433997d3beb","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5df76118a1086433997d3beb: done: 0.014599 sec","job_status":"done","queue_duration_s":0.681691,"scheduling_latency_s":0.681712,"redis_calls":13,"redis_duration_s":0.002966,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000938,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002028,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009662,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014599,"completed_at":"2026-06-29T10:46:05.460Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1662vm-test-run-nixbot-gitlab> gitlab # [  178.643320] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.462Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4f89da5662b888fbe6594b87","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4f89da5662b888fbe6594b87: start","job_status":"start","queue_duration_s":0.697568,"scheduling_latency_s":0.697588}1663vm-test-run-nixbot-gitlab> gitlab # [  178.644896] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.476Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4f89da5662b888fbe6594b87","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4f89da5662b888fbe6594b87: done: 0.014905 sec","job_status":"done","queue_duration_s":0.697568,"scheduling_latency_s":0.697588,"redis_calls":13,"redis_duration_s":0.003412,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001451,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001961,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009513,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014905,"completed_at":"2026-06-29T10:46:05.476Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1664vm-test-run-nixbot-gitlab> gitlab # [  178.742704] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.478Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04ee2f880b49c907e1c81647","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04ee2f880b49c907e1c81647: start","job_status":"start","queue_duration_s":0.713768,"scheduling_latency_s":0.713797}1665vm-test-run-nixbot-gitlab> gitlab # [  178.743929] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.492Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04ee2f880b49c907e1c81647","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04ee2f880b49c907e1c81647: done: 0.01477 sec","job_status":"done","queue_duration_s":0.713768,"scheduling_latency_s":0.713797,"redis_calls":13,"redis_duration_s":0.003045,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001168,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001877,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009528,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01477,"completed_at":"2026-06-29T10:46:05.492Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1666vm-test-run-nixbot-gitlab> gitlab # [  178.828317] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.494Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9817b40c54105ce752ffedd","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9817b40c54105ce752ffedd: start","job_status":"start","queue_duration_s":0.729752,"scheduling_latency_s":0.729772}1667vm-test-run-nixbot-gitlab> gitlab # [  178.829971] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.509Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9817b40c54105ce752ffedd","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9817b40c54105ce752ffedd: done: 0.014895 sec","job_status":"done","queue_duration_s":0.729752,"scheduling_latency_s":0.729772,"redis_calls":13,"redis_duration_s":0.002996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000819,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002177,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010056,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014895,"completed_at":"2026-06-29T10:46:05.509Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1668vm-test-run-nixbot-gitlab> gitlab # [  178.865464] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.510Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8810c6e093c2a3d56357862f","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8810c6e093c2a3d56357862f: start","job_status":"start","queue_duration_s":0.745866,"scheduling_latency_s":0.745886}1669vm-test-run-nixbot-gitlab> gitlab # [  178.939442] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.524Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8810c6e093c2a3d56357862f","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8810c6e093c2a3d56357862f: done: 0.014321 sec","job_status":"done","queue_duration_s":0.745866,"scheduling_latency_s":0.745886,"redis_calls":13,"redis_duration_s":0.002744,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000853,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001891,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009589,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014321,"completed_at":"2026-06-29T10:46:05.524Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1670vm-test-run-nixbot-gitlab> gitlab # [  178.946073] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.525Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0258e048bb6575172a86de21","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0258e048bb6575172a86de21: start","job_status":"start","queue_duration_s":0.761404,"scheduling_latency_s":0.761425}1671vm-test-run-nixbot-gitlab> gitlab # [  179.016378] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.540Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0258e048bb6575172a86de21","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0258e048bb6575172a86de21: done: 0.014489 sec","job_status":"done","queue_duration_s":0.761404,"scheduling_latency_s":0.761425,"redis_calls":13,"redis_duration_s":0.002745,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000821,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001924,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009657,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014489,"completed_at":"2026-06-29T10:46:05.540Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1672vm-test-run-nixbot-gitlab> gitlab # [  179.026196] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.541Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f3b3cb513c2d5eb0988c353","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f3b3cb513c2d5eb0988c353: start","job_status":"start","queue_duration_s":0.777114,"scheduling_latency_s":0.777134}1673vm-test-run-nixbot-gitlab> gitlab # [  179.110521] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.556Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f3b3cb513c2d5eb0988c353","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f3b3cb513c2d5eb0988c353: done: 0.014638 sec","job_status":"done","queue_duration_s":0.777114,"scheduling_latency_s":0.777134,"redis_calls":13,"redis_duration_s":0.003129,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000923,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002206,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0094,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014638,"completed_at":"2026-06-29T10:46:05.556Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1674vm-test-run-nixbot-gitlab> gitlab # [  179.112224] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.557Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae84eb454af7bea6e84c01f8","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae84eb454af7bea6e84c01f8: start","job_status":"start","queue_duration_s":0.793084,"scheduling_latency_s":0.793105}1675vm-test-run-nixbot-gitlab> gitlab # [  179.126221] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.572Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae84eb454af7bea6e84c01f8","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae84eb454af7bea6e84c01f8: done: 0.014661 sec","job_status":"done","queue_duration_s":0.793084,"scheduling_latency_s":0.793105,"redis_calls":13,"redis_duration_s":0.0029679999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00094,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002028,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009196,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014661,"completed_at":"2026-06-29T10:46:05.572Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1676vm-test-run-nixbot-gitlab> gitlab # [  179.236652] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.573Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"980f35d5f7358a2688949123","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-980f35d5f7358a2688949123: start","job_status":"start","queue_duration_s":0.809007,"scheduling_latency_s":0.809027}1677vm-test-run-nixbot-gitlab> gitlab # [  179.238297] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.587Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"980f35d5f7358a2688949123","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-980f35d5f7358a2688949123: done: 0.014457 sec","job_status":"done","queue_duration_s":0.809007,"scheduling_latency_s":0.809027,"redis_calls":13,"redis_duration_s":0.002997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00097,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002027,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009139,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014457,"completed_at":"2026-06-29T10:46:05.587Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1678vm-test-run-nixbot-gitlab> gitlab # [  179.307541] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.589Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"39a6c49b416a83f65f372bfa","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-39a6c49b416a83f65f372bfa: start","job_status":"start","queue_duration_s":0.824776,"scheduling_latency_s":0.824804}1679vm-test-run-nixbot-gitlab> gitlab # [  179.309425] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.604Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"39a6c49b416a83f65f372bfa","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-39a6c49b416a83f65f372bfa: done: 0.015349 sec","job_status":"done","queue_duration_s":0.824776,"scheduling_latency_s":0.824804,"redis_calls":13,"redis_duration_s":0.003332,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000961,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002371,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009364,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015349,"completed_at":"2026-06-29T10:46:05.604Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1680vm-test-run-nixbot-gitlab> gitlab # [  179.417796] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.605Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7cf28e973c9c7a898f67d53f","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7cf28e973c9c7a898f67d53f: start","job_status":"start","queue_duration_s":0.8414,"scheduling_latency_s":0.841421}1681vm-test-run-nixbot-gitlab> gitlab # [  179.419696] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.620Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7cf28e973c9c7a898f67d53f","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7cf28e973c9c7a898f67d53f: done: 0.014624 sec","job_status":"done","queue_duration_s":0.8414,"scheduling_latency_s":0.841421,"redis_calls":13,"redis_duration_s":0.002895,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000893,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002002,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009241,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014624,"completed_at":"2026-06-29T10:46:05.620Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1682vm-test-run-nixbot-gitlab> gitlab # [  179.433334] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.621Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5c46abfe7e6be353285e2b8a","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5c46abfe7e6be353285e2b8a: start","job_status":"start","queue_duration_s":0.857386,"scheduling_latency_s":0.857407}1683vm-test-run-nixbot-gitlab> gitlab # [  179.454486] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.636Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5c46abfe7e6be353285e2b8a","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5c46abfe7e6be353285e2b8a: done: 0.01463 sec","job_status":"done","queue_duration_s":0.857386,"scheduling_latency_s":0.857407,"redis_calls":13,"redis_duration_s":0.00293,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000876,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002054,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009284,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01463,"completed_at":"2026-06-29T10:46:05.636Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1684vm-test-run-nixbot-gitlab> gitlab # [  179.543579] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.637Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a0f16decfde1e58232851c0e","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a0f16decfde1e58232851c0e: start","job_status":"start","queue_duration_s":0.873284,"scheduling_latency_s":0.873305}1685vm-test-run-nixbot-gitlab> gitlab # [  179.545378] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.652Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a0f16decfde1e58232851c0e","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a0f16decfde1e58232851c0e: done: 0.014566 sec","job_status":"done","queue_duration_s":0.873284,"scheduling_latency_s":0.873305,"redis_calls":13,"redis_duration_s":0.00298,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000884,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002096,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009198,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014566,"completed_at":"2026-06-29T10:46:05.652Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1686vm-test-run-nixbot-gitlab> gitlab # [  179.600148] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.653Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"20d61d8704b727cb0d9da564","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-20d61d8704b727cb0d9da564: start","job_status":"start","queue_duration_s":0.889118,"scheduling_latency_s":0.889139}1687vm-test-run-nixbot-gitlab> gitlab # [  179.668864] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.671Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"20d61d8704b727cb0d9da564","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-20d61d8704b727cb0d9da564: done: 0.018221 sec","job_status":"done","queue_duration_s":0.889118,"scheduling_latency_s":0.889139,"redis_calls":13,"redis_duration_s":0.003026,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000956,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00207,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009405,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018221,"completed_at":"2026-06-29T10:46:05.671Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1688vm-test-run-nixbot-gitlab> gitlab # [  179.681434] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.673Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ff5f1aee1cb09889b1e25c2","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ff5f1aee1cb09889b1e25c2: start","job_status":"start","queue_duration_s":0.908765,"scheduling_latency_s":0.908794}1689vm-test-run-nixbot-gitlab> gitlab # [  179.702669] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.687Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ff5f1aee1cb09889b1e25c2","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ff5f1aee1cb09889b1e25c2: done: 0.014594 sec","job_status":"done","queue_duration_s":0.908765,"scheduling_latency_s":0.908794,"redis_calls":13,"redis_duration_s":0.002912,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000934,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001978,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009225,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014594,"completed_at":"2026-06-29T10:46:05.687Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1690vm-test-run-nixbot-gitlab> gitlab # [  179.780497] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.689Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2a6ee82bd3829bdcedef7b49","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2a6ee82bd3829bdcedef7b49: start","job_status":"start","queue_duration_s":0.924663,"scheduling_latency_s":0.924684}1691vm-test-run-nixbot-gitlab> gitlab # [  179.783484] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.704Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2a6ee82bd3829bdcedef7b49","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2a6ee82bd3829bdcedef7b49: done: 0.01508 sec","job_status":"done","queue_duration_s":0.924663,"scheduling_latency_s":0.924684,"redis_calls":13,"redis_duration_s":0.0031060000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00093,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002176,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009371,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01508,"completed_at":"2026-06-29T10:46:05.704Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1692vm-test-run-nixbot-gitlab> gitlab # [  179.878088] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.705Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a8a17a57412bd0cbb6c4f9b1","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a8a17a57412bd0cbb6c4f9b1: start","job_status":"start","queue_duration_s":0.941025,"scheduling_latency_s":0.941046}1693vm-test-run-nixbot-gitlab> gitlab # [  179.879186] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.720Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a8a17a57412bd0cbb6c4f9b1","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a8a17a57412bd0cbb6c4f9b1: done: 0.014585 sec","job_status":"done","queue_duration_s":0.941025,"scheduling_latency_s":0.941046,"redis_calls":13,"redis_duration_s":0.0030239999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00093,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002094,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00918,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014585,"completed_at":"2026-06-29T10:46:05.720Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1694vm-test-run-nixbot-gitlab> gitlab # [  179.954926] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.721Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"72f9e5d49c8f640c970625da","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-72f9e5d49c8f640c970625da: start","job_status":"start","queue_duration_s":0.956867,"scheduling_latency_s":0.956888}1695vm-test-run-nixbot-gitlab> gitlab # [  179.956273] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.737Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"72f9e5d49c8f640c970625da","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-72f9e5d49c8f640c970625da: done: 0.016468 sec","job_status":"done","queue_duration_s":0.956867,"scheduling_latency_s":0.956888,"redis_calls":13,"redis_duration_s":0.003036,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000994,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002042,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010805,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016468,"completed_at":"2026-06-29T10:46:05.737Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1696vm-test-run-nixbot-gitlab> gitlab # [  180.043251] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.739Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a16ba94a5e12f4fb490623dc","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a16ba94a5e12f4fb490623dc: start","job_status":"start","queue_duration_s":0.974847,"scheduling_latency_s":0.974875}1697vm-test-run-nixbot-gitlab> gitlab # [  180.044886] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.754Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a16ba94a5e12f4fb490623dc","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a16ba94a5e12f4fb490623dc: done: 0.015003 sec","job_status":"done","queue_duration_s":0.974847,"scheduling_latency_s":0.974875,"redis_calls":13,"redis_duration_s":0.0028009999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00089,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001911,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009876,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015003,"completed_at":"2026-06-29T10:46:05.754Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1698vm-test-run-nixbot-gitlab> gitlab # [  180.120046] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.755Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7426edbe7a84ffefcd35e1ef","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7426edbe7a84ffefcd35e1ef: start","job_status":"start","queue_duration_s":0.991061,"scheduling_latency_s":0.991081}1699vm-test-run-nixbot-gitlab> gitlab # [  180.121405] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.772Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7426edbe7a84ffefcd35e1ef","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7426edbe7a84ffefcd35e1ef: done: 0.01656 sec","job_status":"done","queue_duration_s":0.991061,"scheduling_latency_s":0.991081,"redis_calls":13,"redis_duration_s":0.004861,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000909,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003952,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009392,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01656,"completed_at":"2026-06-29T10:46:05.772Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1700vm-test-run-nixbot-gitlab> gitlab # [  180.190186] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.773Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9baa458d3af76d23855d365","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9baa458d3af76d23855d365: start","job_status":"start","queue_duration_s":1.008933,"scheduling_latency_s":1.008955}1701vm-test-run-nixbot-gitlab> gitlab # [  180.191827] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.788Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9baa458d3af76d23855d365","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9baa458d3af76d23855d365: done: 0.014637 sec","job_status":"done","queue_duration_s":1.008933,"scheduling_latency_s":1.008955,"redis_calls":13,"redis_duration_s":0.002979,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000937,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002042,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009204,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014637,"completed_at":"2026-06-29T10:46:05.787Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1702vm-test-run-nixbot-gitlab> gitlab # [  180.263239] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.789Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4b1fad8707b7a3e83e3914c5","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4b1fad8707b7a3e83e3914c5: start","job_status":"start","queue_duration_s":1.024855,"scheduling_latency_s":1.024876}1703vm-test-run-nixbot-gitlab> gitlab # [  180.267518] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.804Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4b1fad8707b7a3e83e3914c5","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4b1fad8707b7a3e83e3914c5: done: 0.014926 sec","job_status":"done","queue_duration_s":1.024855,"scheduling_latency_s":1.024876,"redis_calls":13,"redis_duration_s":0.002965,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000945,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00202,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009229,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014926,"completed_at":"2026-06-29T10:46:05.804Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1704vm-test-run-nixbot-gitlab> gitlab # [  180.340624] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.805Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"45aa6c79d15b865545804df9","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-45aa6c79d15b865545804df9: start","job_status":"start","queue_duration_s":1.04107,"scheduling_latency_s":1.041091}1705vm-test-run-nixbot-gitlab> gitlab # [  180.348776] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.820Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"45aa6c79d15b865545804df9","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-45aa6c79d15b865545804df9: done: 0.014631 sec","job_status":"done","queue_duration_s":1.04107,"scheduling_latency_s":1.041091,"redis_calls":13,"redis_duration_s":0.00311,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000963,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002147,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00922,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014631,"completed_at":"2026-06-29T10:46:05.820Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1706vm-test-run-nixbot-gitlab> gitlab # [  180.467576] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.821Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"031f2bd54412ee96dfc0bfb5","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-031f2bd54412ee96dfc0bfb5: start","job_status":"start","queue_duration_s":1.057018,"scheduling_latency_s":1.057039}1707vm-test-run-nixbot-gitlab> gitlab # [  180.469228] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.837Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"031f2bd54412ee96dfc0bfb5","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-031f2bd54412ee96dfc0bfb5: done: 0.015632 sec","job_status":"done","queue_duration_s":1.057018,"scheduling_latency_s":1.057039,"redis_calls":13,"redis_duration_s":0.003876,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001248,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002628,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009505,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015632,"completed_at":"2026-06-29T10:46:05.837Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1708vm-test-run-nixbot-gitlab> gitlab # [  180.494436] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.839Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"368e474923f5000a87baa363","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-368e474923f5000a87baa363: start","job_status":"start","queue_duration_s":1.075267,"scheduling_latency_s":1.075324}1709vm-test-run-nixbot-gitlab> gitlab # [  180.514496] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.861Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"368e474923f5000a87baa363","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-368e474923f5000a87baa363: done: 0.022269 sec","job_status":"done","queue_duration_s":1.075267,"scheduling_latency_s":1.075324,"redis_calls":13,"redis_duration_s":0.006476000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001951,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010168,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022269,"completed_at":"2026-06-29T10:46:05.861Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1710vm-test-run-nixbot-gitlab> gitlab # [  180.577839] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.866Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"54192af00598099fe29693fb","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-54192af00598099fe29693fb: start","job_status":"start","queue_duration_s":1.101946,"scheduling_latency_s":1.101974}1711vm-test-run-nixbot-gitlab> gitlab # [  180.597585] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.888Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"54192af00598099fe29693fb","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-54192af00598099fe29693fb: done: 0.022519 sec","job_status":"done","queue_duration_s":1.101946,"scheduling_latency_s":1.101974,"redis_calls":13,"redis_duration_s":0.008836,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002286,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00655,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01016,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022519,"completed_at":"2026-06-29T10:46:05.888Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1712vm-test-run-nixbot-gitlab> gitlab # [  180.671334] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.890Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e913260b72b6f0c1c988cc6c","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e913260b72b6f0c1c988cc6c: start","job_status":"start","queue_duration_s":1.125905,"scheduling_latency_s":1.125927}1713vm-test-run-nixbot-gitlab> gitlab # [  180.677810] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.905Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e913260b72b6f0c1c988cc6c","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e913260b72b6f0c1c988cc6c: done: 0.015055 sec","job_status":"done","queue_duration_s":1.125905,"scheduling_latency_s":1.125927,"redis_calls":13,"redis_duration_s":0.003039,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000876,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002163,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009309,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015055,"completed_at":"2026-06-29T10:46:05.905Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1714vm-test-run-nixbot-gitlab> gitlab # [  180.758707] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.906Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1409ce4832a9f9b70a2199e0","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1409ce4832a9f9b70a2199e0: start","job_status":"start","queue_duration_s":1.142228,"scheduling_latency_s":1.142248}1715vm-test-run-nixbot-gitlab> gitlab # [  180.760495] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.921Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1409ce4832a9f9b70a2199e0","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1409ce4832a9f9b70a2199e0: done: 0.014672 sec","job_status":"done","queue_duration_s":1.142228,"scheduling_latency_s":1.142248,"redis_calls":13,"redis_duration_s":0.003052,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00091,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002142,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00922,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014672,"completed_at":"2026-06-29T10:46:05.921Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1716vm-test-run-nixbot-gitlab> gitlab # [  180.864507] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.922Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"90dbc71e28d93f66f71076da","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-90dbc71e28d93f66f71076da: start","job_status":"start","queue_duration_s":1.158169,"scheduling_latency_s":1.158189}1717vm-test-run-nixbot-gitlab> gitlab # [  180.866411] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.937Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"90dbc71e28d93f66f71076da","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-90dbc71e28d93f66f71076da: done: 0.014688 sec","job_status":"done","queue_duration_s":1.158169,"scheduling_latency_s":1.158189,"redis_calls":13,"redis_duration_s":0.003071,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000927,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002144,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009268,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014688,"completed_at":"2026-06-29T10:46:05.937Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1718vm-test-run-nixbot-gitlab> gitlab # [  180.906196] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.938Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f6bbd5d9a01473cc3cf5c72d","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f6bbd5d9a01473cc3cf5c72d: start","job_status":"start","queue_duration_s":1.174118,"scheduling_latency_s":1.174139}1719vm-test-run-nixbot-gitlab> gitlab # [  180.926108] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.953Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f6bbd5d9a01473cc3cf5c72d","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f6bbd5d9a01473cc3cf5c72d: done: 0.014635 sec","job_status":"done","queue_duration_s":1.174118,"scheduling_latency_s":1.174139,"redis_calls":13,"redis_duration_s":0.002933,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00092,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002013,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009226,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014635,"completed_at":"2026-06-29T10:46:05.953Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1720vm-test-run-nixbot-gitlab> gitlab # [  181.016292] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.954Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9a24ee84d48a3224ef225d3c","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9a24ee84d48a3224ef225d3c: start","job_status":"start","queue_duration_s":1.190038,"scheduling_latency_s":1.190059}1721vm-test-run-nixbot-gitlab> gitlab # [  181.017890] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.969Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9a24ee84d48a3224ef225d3c","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9a24ee84d48a3224ef225d3c: done: 0.014512 sec","job_status":"done","queue_duration_s":1.190038,"scheduling_latency_s":1.190059,"redis_calls":13,"redis_duration_s":0.002902,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000938,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001964,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009609,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014512,"completed_at":"2026-06-29T10:46:05.968Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1722vm-test-run-nixbot-gitlab> gitlab # [  181.108127] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.970Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4216954cc3bbdaf1cbdf8875","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4216954cc3bbdaf1cbdf8875: start","job_status":"start","queue_duration_s":1.205829,"scheduling_latency_s":1.20585}1723vm-test-run-nixbot-gitlab> gitlab # [  181.109489] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.984Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4216954cc3bbdaf1cbdf8875","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4216954cc3bbdaf1cbdf8875: done: 0.01444 sec","job_status":"done","queue_duration_s":1.205829,"scheduling_latency_s":1.20585,"redis_calls":13,"redis_duration_s":0.002835,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00091,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001925,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009554,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01444,"completed_at":"2026-06-29T10:46:05.984Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1724vm-test-run-nixbot-gitlab> gitlab # [  181.179286] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:05.986Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c124d92851ef7d674eebb7f","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c124d92851ef7d674eebb7f: start","job_status":"start","queue_duration_s":1.221572,"scheduling_latency_s":1.221592}1725vm-test-run-nixbot-gitlab> gitlab # [  181.180471] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:06.000Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c124d92851ef7d674eebb7f","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c124d92851ef7d674eebb7f: done: 0.014735 sec","job_status":"done","queue_duration_s":1.221572,"scheduling_latency_s":1.221592,"redis_calls":13,"redis_duration_s":0.0028919999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000923,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001969,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009626,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014735,"completed_at":"2026-06-29T10:46:06.000Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1726vm-test-run-nixbot-gitlab> gitlab # [  181.249959] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:06.002Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"05d74d4fa9a72f15a320eaf6","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-05d74d4fa9a72f15a320eaf6: start","job_status":"start","queue_duration_s":1.237584,"scheduling_latency_s":1.237604}1727vm-test-run-nixbot-gitlab> gitlab # [  181.251117] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:06.016Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"05d74d4fa9a72f15a320eaf6","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-05d74d4fa9a72f15a320eaf6: done: 0.014605 sec","job_status":"done","queue_duration_s":1.237584,"scheduling_latency_s":1.237604,"redis_calls":13,"redis_duration_s":0.002946,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000904,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002042,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009662,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014605,"completed_at":"2026-06-29T10:46:06.016Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1728vm-test-run-nixbot-gitlab> gitlab # [  181.354583] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:06.017Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b8d9abee5703e8aab18b783","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b8d9abee5703e8aab18b783: start","job_status":"start","queue_duration_s":1.253431,"scheduling_latency_s":1.253484}1729vm-test-run-nixbot-gitlab> gitlab # [  181.356124] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:06.036Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b8d9abee5703e8aab18b783","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b8d9abee5703e8aab18b783: done: 0.01878 sec","job_status":"done","queue_duration_s":1.253431,"scheduling_latency_s":1.253484,"redis_calls":13,"redis_duration_s":0.006998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001289,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005709,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009516,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01878,"completed_at":"2026-06-29T10:46:06.036Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1730vm-test-run-nixbot-gitlab> gitlab # [  181.459821] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:06.038Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f2d0961cc23cee35dccfce4a","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f2d0961cc23cee35dccfce4a: start","job_status":"start","queue_duration_s":1.273549,"scheduling_latency_s":1.273572}1731vm-test-run-nixbot-gitlab> gitlab # [  181.461724] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:06.052Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f2d0961cc23cee35dccfce4a","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f2d0961cc23cee35dccfce4a: done: 0.014779 sec","job_status":"done","queue_duration_s":1.273549,"scheduling_latency_s":1.273572,"redis_calls":13,"redis_duration_s":0.003118,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000915,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002203,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009298,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014779,"completed_at":"2026-06-29T10:46:06.052Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1732vm-test-run-nixbot-gitlab> gitlab # [  181.478641] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:06.054Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"01bfebbe64efcb9be68ac837","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-01bfebbe64efcb9be68ac837: start","job_status":"start","queue_duration_s":1.28963,"scheduling_latency_s":1.289651}1733vm-test-run-nixbot-gitlab> gitlab # [  181.499276] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:06.069Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"01bfebbe64efcb9be68ac837","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-01bfebbe64efcb9be68ac837: done: 0.015062 sec","job_status":"done","queue_duration_s":1.28963,"scheduling_latency_s":1.289651,"redis_calls":13,"redis_duration_s":0.003304,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000886,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002418,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009461,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015062,"completed_at":"2026-06-29T10:46:06.069Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1734vm-test-run-nixbot-gitlab> gitlab # [  181.571496] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:06.070Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"90254272916d244581e6eaab","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-90254272916d244581e6eaab: start","job_status":"start","queue_duration_s":1.305965,"scheduling_latency_s":1.305986}1735vm-test-run-nixbot-gitlab> gitlab # [  181.580727] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:06.084Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"90254272916d244581e6eaab","created_at":"2026-06-29T10:46:04.689Z","trace_propagation_headers":{"sentry-trace":"b851f54eaa8c459e9fc30ce069af37bc-035e8e34b63d43a5","baggage":"sentry-trace_id=b851f54eaa8c459e9fc30ce069af37bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a5b30dd49e80eb38f528971c996d02d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:04.764Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-90254272916d244581e6eaab: done: 0.014568 sec","job_status":"done","queue_duration_s":1.305965,"scheduling_latency_s":1.305986,"redis_calls":13,"redis_duration_s":0.002923,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000914,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002009,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009152,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014568,"completed_at":"2026-06-29T10:46:06.084Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1736vm-test-run-nixbot-gitlab> gitlab # [  191.917564] runuser[2699]: pam_unix(runuser:session): session closed for user gitlab1737vm-test-run-nixbot-gitlab> gitlab # [  191.924219] setup-gitlab-start[2913]: ++ jq -r .token1738vm-test-run-nixbot-gitlab> gitlab # [  191.927987] setup-gitlab-start[2914]: +++ date -d '+7 days' +%F1739vm-test-run-nixbot-gitlab> gitlab # [  191.934943] setup-gitlab-start[2912]: ++ curl -fs -X POST -H 'Authorization: Bearer 4d02a22efa6e41be75d30815dc81239bb7cbcca6cbf6d06438eaa5a7bd14df09' -H 'Content-Type: application/json' -d '{"name":"nixbot","scopes":["api"],"expires_at":"2026-07-06"}' http://gitlab/api/v4/users/1/personal_access_tokens1740vm-test-run-nixbot-gitlab> gitlab # [  192.271345] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:20 +0000] "POST /api/v4/users/1/personal_access_tokens HTTP/1.1" 201 267 "" "curl/8.20.0" 3141741vm-test-run-nixbot-gitlab> gitlab # [  192.280574] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:20.243Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"733342ff545aafa6a4203867","created_at":"2026-06-29T10:46:20.212Z","trace_propagation_headers":{"sentry-trace":"b4031b97d6fc411a9c92532298b9a4b2-70a8b31c63be4abc","baggage":"sentry-trace_id=b4031b97d6fc411a9c92532298b9a4b2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9FSE0RVKMH7Z4NF5SP0Z7W","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:fe41a6587ca506fd951b41c6e2be51872654aa5346bbe1f9601e208ec67e3fd1","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:20.235Z","job_size_bytes":451,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-733342ff545aafa6a4203867: start","job_status":"start","queue_duration_s":0.007779,"scheduling_latency_s":0.007844}1742vm-test-run-nixbot-gitlab> gitlab # [  192.337084] setup-gitlab-start[2651]: + TOKEN=glpat--Dbe5iUDzWz5yzlhrAWYaG86MQp1OjEH.01.0w10m6ofy1743vm-test-run-nixbot-gitlab> gitlab # [  192.339193] setup-gitlab-start[2651]: + test -n glpat--Dbe5iUDzWz5yzlhrAWYaG86MQp1OjEH.01.0w10m6ofy1744vm-test-run-nixbot-gitlab> gitlab # [  192.341278] setup-gitlab-start[2651]: + test glpat--Dbe5iUDzWz5yzlhrAWYaG86MQp1OjEH.01.0w10m6ofy '!=' null1745vm-test-run-nixbot-gitlab> gitlab # [  192.343505] setup-gitlab-start[2651]: + mkdir -p /var/lib/secrets1746vm-test-run-nixbot-gitlab> gitlab # [  192.344974] setup-gitlab-start[2651]: + echo glpat--Dbe5iUDzWz5yzlhrAWYaG86MQp1OjEH.01.0w10m6ofy1747vm-test-run-nixbot-gitlab> gitlab # [  192.347235] setup-gitlab-start[2651]: + chmod 644 /var/lib/secrets/gitlab-token1748vm-test-run-nixbot-gitlab> gitlab # [  192.348991] setup-gitlab-start[2651]: + curl -fs -X POST -H 'PRIVATE-TOKEN: glpat--Dbe5iUDzWz5yzlhrAWYaG86MQp1OjEH.01.0w10m6ofy' -H 'Content-Type: application/json' -d '{"name":"test-flake","visibility":"public","topics":["build-with-buildbot"]}' http://gitlab/api/v4/projects1749vm-test-run-nixbot-gitlab> gitlab # [  192.401923] sidekiq-cluster[2114]: {"severity":"WARN","time":"2026-06-29T10:46:20.374Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"733342ff545aafa6a4203867","created_at":"2026-06-29T10:46:20.212Z","trace_propagation_headers":{"sentry-trace":"b4031b97d6fc411a9c92532298b9a4b2-70a8b31c63be4abc","baggage":"sentry-trace_id=b4031b97d6fc411a9c92532298b9a4b2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9FSE0RVKMH7Z4NF5SP0Z7W","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:fe41a6587ca506fd951b41c6e2be51872654aa5346bbe1f9601e208ec67e3fd1","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:20.235Z","job_size_bytes":451,"pid":2114,"sidekiq_tid":"lynm","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-733342ff545aafa6a4203867: fail: 0.131205 sec","job_status":"fail","queue_duration_s":0.007779,"scheduling_latency_s":0.007844,"redis_calls":11,"redis_duration_s":0.004426,"redis_read_bytes":137,"redis_write_bytes":1317,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001305,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002305,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000816,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.077648,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000641143,"duration_s":0.131205,"completed_at":"2026-06-29T10:46:20.374Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.038857,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1750vm-test-run-nixbot-gitlab> gitlab # [  193.480155] gitaly[1389]: time="2026-06-29T10:46:21.452Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSEBZRVWCH408WVKVMNT0 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T10:46:31.011" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T10:46:21.452" pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1751vm-test-run-nixbot-gitlab> gitlab # [  193.483682] gitaly[1389]: time="2026-06-29T10:46:21.453Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSEBZRVWCH408WVKVMNT0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T10:46:31.011" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T10:46:21.452" grpc.time_ms=0.724 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1752vm-test-run-nixbot-gitlab> gitlab # [  193.615983] gitaly[1389]: time="2026-06-29T10:46:21.589Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSEBZRVWCH408WVKVMNT0 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T10:46:31.148" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T10:46:21.589" pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1753vm-test-run-nixbot-gitlab> gitlab # [  193.619551] gitaly[1389]: time="2026-06-29T10:46:21.589Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSEBZRVWCH408WVKVMNT0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T10:46:31.148" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T10:46:21.589" grpc.time_ms=0.24 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1754vm-test-run-nixbot-gitlab> gitlab # [  193.717280] gitaly[1389]: time="2026-06-29T10:46:21.690Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSEBZRVWCH408WVKVMNT0 diskcache=ac9882c2-31ce-4c19-8c3b-b04cd153955a grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-29T10:46:31.220" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T10:46:21.660" pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1755vm-test-run-nixbot-gitlab> gitlab # [  193.719926] gitaly[1389]: time="2026-06-29T10:46:21.691Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=326 command.oublock=32 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSEBZRVWCH408WVKVMNT0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-29T10:46:31.220" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T10:46:21.660" grpc.time_ms=30.337 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1756vm-test-run-nixbot-gitlab> gitlab # [  193.777383] gitaly[1389]: time="2026-06-29T10:46:21.750Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSEBZRVWCH408WVKVMNT0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T10:46:31.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T10:46:21.749" pid=1389 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1757vm-test-run-nixbot-gitlab> gitlab # [  193.779368] gitaly[1389]: time="2026-06-29T10:46:21.751Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSEBZRVWCH408WVKVMNT0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T10:46:31.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T10:46:21.749" grpc.time_ms=1.027 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1758vm-test-run-nixbot-gitlab> gitlab # [  193.794546] gitaly[1389]: time="2026-06-29T10:46:21.768Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=620 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSEBZRVWCH408WVKVMNT0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-29T10:46:31.014" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T10:46:21.756" grpc.time_ms=11.51 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1759vm-test-run-nixbot-gitlab> gitlab # [  193.869823] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:21.842Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"02faf61e08b53a9a4c2b0037","created_at":"2026-06-29T10:46:21.821Z","trace_propagation_headers":{"sentry-trace":"1cae006fd816450db210000c79b8c75d-83acfa57b4124998","baggage":"sentry-trace_id=1cae006fd816450db210000c79b8c75d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9FSEBZRVWCH408WVKVMNT0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:21.839Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-02faf61e08b53a9a4c2b0037: start","job_status":"start","queue_duration_s":0.002769,"scheduling_latency_s":0.002834}1760vm-test-run-nixbot-gitlab> gitlab # [  193.894896] gitaly[1389]: time="2026-06-29T10:46:21.860Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSEBZRVWCH408WVKVMNT0 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T10:46:31.119" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T10:46:21.860" pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1761vm-test-run-nixbot-gitlab> gitlab # [  193.896587] gitaly[1389]: time="2026-06-29T10:46:21.861Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSEBZRVWCH408WVKVMNT0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T10:46:31.119" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T10:46:21.860" grpc.time_ms=0.607 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1762vm-test-run-nixbot-gitlab> gitlab # [  193.922859] gitaly[1389]: time="2026-06-29T10:46:21.896Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSEBZRVWCH408WVKVMNT0 diskcache=6427a801-1cb3-4536-99bf-90d5830776b5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-29T10:46:31.124" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T10:46:21.865" pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1763vm-test-run-nixbot-gitlab> gitlab # [  193.924316] gitaly[1389]: time="2026-06-29T10:46:21.897Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=329 command.oublock=32 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSEBZRVWCH408WVKVMNT0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-29T10:46:31.124" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T10:46:21.865" grpc.time_ms=31.416 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1764vm-test-run-nixbot-gitlab> gitlab # [  193.931292] gitaly[1389]: time="2026-06-29T10:46:21.905Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSEBZRVWCH408WVKVMNT0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T10:46:31.163" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T10:46:21.904" pid=1389 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1765vm-test-run-nixbot-gitlab> gitlab # [  193.932409] gitaly[1389]: time="2026-06-29T10:46:21.905Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSEBZRVWCH408WVKVMNT0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T10:46:31.163" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T10:46:21.904" grpc.time_ms=0.971 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1766vm-test-run-nixbot-gitlab> gitlab # [  193.944741] gitaly[1389]: time="2026-06-29T10:46:21.918Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSEBZRVWCH408WVKVMNT0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-29T10:46:31.171" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T10:46:21.912" grpc.time_ms=6.054 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1767vm-test-run-nixbot-gitlab> gitlab # [  193.978205] gitaly[1389]: time="2026-06-29T10:46:21.951Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSEBZRVWCH408WVKVMNT0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T10:46:31.211" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T10:46:21.951" grpc.time_ms=0.123 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1768vm-test-run-nixbot-gitlab> gitlab # [  193.983940] gitaly[1389]: time="2026-06-29T10:46:21.957Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSEBZRVWCH408WVKVMNT0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-29T10:46:31.217" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T10:46:21.957" grpc.time_ms=0.157 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1769vm-test-run-nixbot-gitlab> gitlab # [  194.105122] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:22.078Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"02faf61e08b53a9a4c2b0037","created_at":"2026-06-29T10:46:21.821Z","trace_propagation_headers":{"sentry-trace":"1cae006fd816450db210000c79b8c75d-83acfa57b4124998","baggage":"sentry-trace_id=1cae006fd816450db210000c79b8c75d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9FSEBZRVWCH408WVKVMNT0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:21.839Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-02faf61e08b53a9a4c2b0037: done: 0.236188 sec","job_status":"done","queue_duration_s":0.002769,"scheduling_latency_s":0.002834,"redis_calls":10,"redis_duration_s":0.003745,"redis_read_bytes":8,"redis_write_bytes":1165,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002096,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001649,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":7,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.174,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.17,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.051028,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.203199569,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.236188,"completed_at":"2026-06-29T10:46:22.078Z","load_balancing_strategy":"primary","db_duration_s":0.173661,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1770vm-test-run-nixbot-gitlab> gitlab # [  194.174880] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:22.148Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"53ebfec1b35574135ccf40b7","created_at":"2026-06-29T10:46:21.846Z","trace_propagation_headers":{"sentry-trace":"1cae006fd816450db210000c79b8c75d-83acfa57b4124998","baggage":"sentry-trace_id=1cae006fd816450db210000c79b8c75d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9FSEBZRVWCH408WVKVMNT0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:21.854Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-53ebfec1b35574135ccf40b7: start","job_status":"start","queue_duration_s":0.293276,"scheduling_latency_s":0.293313}1771vm-test-run-nixbot-gitlab> gitlab # [  194.252601] gitaly[1389]: time="2026-06-29T10:46:22.226Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSEBZRVWCH408WVKVMNT0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-29T10:46:32.008" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T10:46:22.218" grpc.time_ms=7.085 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1772vm-test-run-nixbot-gitlab> gitlab # [  194.284740] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:22.258Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"53ebfec1b35574135ccf40b7","created_at":"2026-06-29T10:46:21.846Z","trace_propagation_headers":{"sentry-trace":"1cae006fd816450db210000c79b8c75d-83acfa57b4124998","baggage":"sentry-trace_id=1cae006fd816450db210000c79b8c75d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9FSEBZRVWCH408WVKVMNT0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:21.854Z","job_size_bytes":2,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-53ebfec1b35574135ccf40b7: done: 0.109824 sec","job_status":"done","queue_duration_s":0.293276,"scheduling_latency_s":0.293313,"redis_calls":14,"redis_duration_s":0.003956,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000733,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002007,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001216,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.049449,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.063387385,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.109824,"completed_at":"2026-06-29T10:46:22.257Z","load_balancing_strategy":"primary","db_duration_s":0.028207,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1773vm-test-run-nixbot-gitlab> gitlab # [  194.357430] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:22.330Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"36e95ae08105088ab6bf702f","created_at":"2026-06-29T10:46:22.136Z","trace_propagation_headers":{"sentry-trace":"1cae006fd816450db210000c79b8c75d-83acfa57b4124998","baggage":"sentry-trace_id=1cae006fd816450db210000c79b8c75d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9FSEBZRVWCH408WVKVMNT0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:22.174Z","job_size_bytes":5,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-36e95ae08105088ab6bf702f: start","job_status":"start","queue_duration_s":0.155922,"scheduling_latency_s":0.155983}1774vm-test-run-nixbot-gitlab> gitlab # [  194.453390] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:22.426Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"36e95ae08105088ab6bf702f","created_at":"2026-06-29T10:46:22.136Z","trace_propagation_headers":{"sentry-trace":"1cae006fd816450db210000c79b8c75d-83acfa57b4124998","baggage":"sentry-trace_id=1cae006fd816450db210000c79b8c75d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9FSEBZRVWCH408WVKVMNT0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:22.174Z","job_size_bytes":5,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-36e95ae08105088ab6bf702f: done: 0.096389 sec","job_status":"done","queue_duration_s":0.155922,"scheduling_latency_s":0.155983,"redis_calls":12,"redis_duration_s":0.003731,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001106,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001993,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000632,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.063696,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000375746,"duration_s":0.096389,"completed_at":"2026-06-29T10:46:22.426Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013199,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1775vm-test-run-nixbot-gitlab> gitlab # [  194.527795] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:22.500Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"3c63047a0417a69a13948fc5","created_at":"2026-06-29T10:46:22.197Z","trace_propagation_headers":{"sentry-trace":"1cae006fd816450db210000c79b8c75d-83acfa57b4124998","baggage":"sentry-trace_id=1cae006fd816450db210000c79b8c75d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9FSEBZRVWCH408WVKVMNT0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:22.201Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-3c63047a0417a69a13948fc5: start","job_status":"start","queue_duration_s":0.299181,"scheduling_latency_s":0.299245}1776vm-test-run-nixbot-gitlab> gitlab # [  194.620756] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:22 +0000] "POST /api/v4/projects HTTP/1.1" 201 4795 "" "curl/8.20.0" 23051777vm-test-run-nixbot-gitlab> gitlab # [  194.625724] setup-gitlab-start[2651]: + mkdir -p /tmp/test-flake1778vm-test-run-nixbot-gitlab> gitlab # [  194.628793] setup-gitlab-start[2651]: + cd /tmp/test-flake1779vm-test-run-nixbot-gitlab> gitlab # [  194.630613] setup-gitlab-start[2651]: + git init -b master1780vm-test-run-nixbot-gitlab> gitlab # [  194.653628] setup-gitlab-start[2957]: Initialized empty Git repository in /tmp/test-flake/.git/1781vm-test-run-nixbot-gitlab> gitlab # [  194.655002] setup-gitlab-start[2651]: + git config user.name test1782vm-test-run-nixbot-gitlab> gitlab # [  194.658872] setup-gitlab-start[2651]: + git config user.email test@example.com1783vm-test-run-nixbot-gitlab> gitlab # [  194.662847] setup-gitlab-start[2651]: + cat1784vm-test-run-nixbot-gitlab> gitlab # [  194.665742] setup-gitlab-start[2651]: + git add flake.nix1785vm-test-run-nixbot-gitlab> gitlab # [  194.669536] setup-gitlab-start[2651]: + git commit -m 'initial commit'1786vm-test-run-nixbot-gitlab> gitlab # [  194.677377] setup-gitlab-start[2962]: [master (root-commit) f55b93e] initial commit1787vm-test-run-nixbot-gitlab> gitlab # [  194.678414] setup-gitlab-start[2962]:  1 file changed, 10 insertions(+)1788vm-test-run-nixbot-gitlab> gitlab # [  194.679625] setup-gitlab-start[2962]:  create mode 100644 flake.nix1789vm-test-run-nixbot-gitlab> gitlab # [  194.680441] setup-gitlab-start[2651]: + git remote add origin http://oauth2:glpat--Dbe5iUDzWz5yzlhrAWYaG86MQp1OjEH.01.0w10m6ofy@gitlab/root/test-flake.git1790vm-test-run-nixbot-gitlab> gitlab # [  194.687954] setup-gitlab-start[2651]: + git push -u origin master1791vm-test-run-nixbot-gitlab> gitlab # [  194.721181] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:22.694Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"3c63047a0417a69a13948fc5","created_at":"2026-06-29T10:46:22.197Z","trace_propagation_headers":{"sentry-trace":"1cae006fd816450db210000c79b8c75d-83acfa57b4124998","baggage":"sentry-trace_id=1cae006fd816450db210000c79b8c75d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9FSEBZRVWCH408WVKVMNT0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:22.201Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-3c63047a0417a69a13948fc5: done: 0.193753 sec","job_status":"done","queue_duration_s":0.299181,"scheduling_latency_s":0.299245,"redis_calls":11,"redis_duration_s":0.005444,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001732,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002748,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000964,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.136,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.136,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.127417,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000389435,"duration_s":0.193753,"completed_at":"2026-06-29T10:46:22.694Z","load_balancing_strategy":"primary","db_duration_s":0.064646,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1792vm-test-run-nixbot-gitlab> gitlab # [  194.725992] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:22.698Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"9291ccbbbd23851fec49578a","created_at":"2026-06-29T10:46:22.203Z","trace_propagation_headers":{"sentry-trace":"1cae006fd816450db210000c79b8c75d-83acfa57b4124998","baggage":"sentry-trace_id=1cae006fd816450db210000c79b8c75d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9FSEBZRVWCH408WVKVMNT0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:22.207Z","job_size_bytes":89,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-9291ccbbbd23851fec49578a: start","job_status":"start","queue_duration_s":0.490253,"scheduling_latency_s":0.490276}1793vm-test-run-nixbot-gitlab> gitlab # [  194.783135] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:22.756Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"9291ccbbbd23851fec49578a","created_at":"2026-06-29T10:46:22.203Z","trace_propagation_headers":{"sentry-trace":"1cae006fd816450db210000c79b8c75d-83acfa57b4124998","baggage":"sentry-trace_id=1cae006fd816450db210000c79b8c75d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9FSEBZRVWCH408WVKVMNT0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:22.207Z","job_size_bytes":89,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-9291ccbbbd23851fec49578a: done: 0.058621 sec","job_status":"done","queue_duration_s":0.490253,"scheduling_latency_s":0.490276,"redis_calls":11,"redis_duration_s":0.007791,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.003637,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003658,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000496,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025746,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000398375,"duration_s":0.058621,"completed_at":"2026-06-29T10:46:22.756Z","load_balancing_strategy":"primary","db_duration_s":0.020393,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1794vm-test-run-nixbot-gitlab> gitlab # [  194.854062] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:22 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 911795vm-test-run-nixbot-gitlab> gitlab # [  194.943344] gitaly[1389]: time="2026-06-29T10:46:22.916Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSGSPEK2XW6S10A3434B0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T10:46:22.908" grpc.time_ms=8.195 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1796vm-test-run-nixbot-gitlab> gitlab # [  194.965920] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:22 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.54.0" 1641797vm-test-run-nixbot-gitlab> gitlab # [  195.191993] gitaly[1389]: time="2026-06-29T10:46:23.165Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSGZ651EC2T4H8V643V3V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-29T10:46:33.001" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T10:46:23.152" grpc.time_ms=12.758 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1798vm-test-run-nixbot-gitlab> gitlab # [  195.267496] gitaly[1389]: time="2026-06-29T10:46:23.240Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=328 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSGZ651EC2T4H8V643V3V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-06-29T10:46:53.033" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-06-29T10:46:23.234" grpc.time_ms=6.635 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1799vm-test-run-nixbot-gitlab> gitlab # [  195.317133] gitaly[1389]: time="2026-06-29T10:46:23.290Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KW9FSGZ651EC2T4H8V643V3V duration_ms=199ns method=POST pid=1389 status=200 url="http://unix/api/v4/internal/allowed"1800vm-test-run-nixbot-gitlab> gitlab # [  195.375580] gitaly[1389]: time="2026-06-29T10:46:23.349Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KW9FSGZ651EC2T4H8V643V3V duration_ms=52ns method=POST pid=1389 status=200 url="http://unix/api/v4/internal/pre_receive"1801vm-test-run-nixbot-gitlab> gitlab # [  195.381059] gitaly[1389]: time="2026-06-29T10:46:23.354Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9FSGZ651EC2T4H8V643V3V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5319 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T10:46:23.090" grpc.time_ms=264.281 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1802vm-test-run-nixbot-gitlab> gitlab # [  195.429579] gitaly[1389]: time="2026-06-29T10:46:23.400Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9FSGZ651EC2T4H8V643V3V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4789 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T10:46:23.395" grpc.time_ms=4.238 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1803vm-test-run-nixbot-gitlab> gitlab # [  195.468793] gitaly[1389]: time="2026-06-29T10:46:23.442Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9FSGZ651EC2T4H8V643V3V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4788 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T10:46:23.442" grpc.time_ms=0.371 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1804vm-test-run-nixbot-gitlab> gitlab # [  195.508836] gitaly[1389]: time="2026-06-29T10:46:23.482Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9FSGZ651EC2T4H8V643V3V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4790 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T10:46:23.482" grpc.time_ms=0.405 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1805vm-test-run-nixbot-gitlab> gitlab # [  195.615719] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:23.588Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 f55b93e3be80c71b3412c6f12756acbfb61aa22b refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"28741fa07f143cdae31f3c99","created_at":"2026-06-29T10:46:23.574Z","trace_propagation_headers":{"sentry-trace":"0137635c5bf24646accde69c40ec75da-a73b6f95ed7645e7","baggage":"sentry-trace_id=0137635c5bf24646accde69c40ec75da,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9FSGZ651EC2T4H8V643V3V","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:23.578Z","job_size_bytes":185,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-28741fa07f143cdae31f3c99: start","job_status":"start","queue_duration_s":0.010137,"scheduling_latency_s":0.010171}1806vm-test-run-nixbot-gitlab> gitlab # [  195.663477] gitaly[1389]: time="2026-06-29T10:46:23.637Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSGZ651EC2T4H8V643V3V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-29T10:46:33.188" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T10:46:23.628" grpc.time_ms=8.325 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1807vm-test-run-nixbot-gitlab> gitlab # [  195.688597] gitaly[1389]: time="2026-06-29T10:46:23.662Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1567 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=4 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSGZ651EC2T4H8V643V3V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-06-29T10:46:33.204" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T10:46:23.645" grpc.time_ms=16.741 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1808vm-test-run-nixbot-gitlab> gitlab # [  195.707563] gitaly[1389]: time="2026-06-29T10:46:23.679Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KW9FSGZ651EC2T4H8V643V3V duration_ms=151ns method=POST pid=1389 status=200 url="http://unix/api/v4/internal/post_receive"1809vm-test-run-nixbot-gitlab> gitlab # [  195.709200] gitaly[1389]: time="2026-06-29T10:46:23.681Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9FSGZ651EC2T4H8V643V3V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4813 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T10:46:23.527" grpc.time_ms=153.031 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1810vm-test-run-nixbot-gitlab> gitlab # [  195.743440] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:23.716Z","message":"changes: \"0000000000000000000000000000000000000000 f55b93e3be80c71b3412c6f12756acbfb61aa22b refs/heads/master\\n\""}1811vm-test-run-nixbot-gitlab> gitlab # [  195.744807] gitaly[1389]: time="2026-06-29T10:46:23.716Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KW9FSGZ651EC2T4H8V643V3V diskcache=f886d72b-59ff-47ee-a8b3-38bdd20560ba grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T10:46:23.006" pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1812vm-test-run-nixbot-gitlab> gitlab # [  195.745985] gitaly[1389]: time="2026-06-29T10:46:23.717Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=191 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13212 command.oublock=32 command.real_time_ms=697 command.spawn_token_fork_ms=1 command.system_time_ms=124 command.user_time_ms=67 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSGZ651EC2T4H8V643V3V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=684 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T10:46:23.006" grpc.time_ms=710.246 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1813vm-test-run-nixbot-gitlab> gitlab # [  195.748333] gitlab-workhorse[1291]: time="2026-06-29T10:46:23Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521814vm-test-run-nixbot-gitlab> gitlab # [  195.749294] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:23 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 7681815vm-test-run-nixbot-gitlab> gitlab # [  195.791047] setup-gitlab-start[2966]: To http://gitlab/root/test-flake.git1816vm-test-run-nixbot-gitlab> gitlab # [  195.792454] setup-gitlab-start[2966]:  * [new branch]      master -> master1817vm-test-run-nixbot-gitlab> gitlab # [  195.793925] setup-gitlab-start[2966]: branch 'master' set up to track 'origin/master'.1818vm-test-run-nixbot-gitlab> gitlab # [  195.797497] systemd[1]: Finished setup-gitlab.service.1819vm-test-run-nixbot-gitlab> gitlab # [  195.800555] systemd[1]: Starting nixbot CI service...1820vm-test-run-nixbot-gitlab> gitlab # [  195.833661] gitaly[1389]: time="2026-06-29T10:46:23.807Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSGZ651EC2T4H8V643V3V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T10:46:33.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T10:46:23.798" grpc.time_ms=8.365 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1821vm-test-run-nixbot-gitlab> gitlab # [  195.872061] gitaly[1389]: time="2026-06-29T10:46:23.845Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSGZ651EC2T4H8V643V3V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-29T10:46:33.048" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T10:46:23.839" grpc.time_ms=6.136 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1822vm-test-run-nixbot-gitlab> gitlab # [  195.932738] gitaly[1389]: time="2026-06-29T10:46:23.906Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSGZ651EC2T4H8V643V3V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-06-29T10:46:33.107" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=226 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T10:46:23.897" grpc.time_ms=8.764 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1823vm-test-run-nixbot-gitlab> gitlab # [  196.026689] gitaly[1389]: time="2026-06-29T10:46:24.000Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9FSGZ651EC2T4H8V643V3V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3940 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T10:46:23.999" grpc.time_ms=0.486 pid=1389 span.kind=server system=grpc1824vm-test-run-nixbot-gitlab> gitlab # [  196.065738] gitaly[1389]: time="2026-06-29T10:46:24.039Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9FSGZ651EC2T4H8V643V3V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3942 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T10:46:24.038" grpc.time_ms=0.364 pid=1389 span.kind=server system=grpc1825vm-test-run-nixbot-gitlab> gitlab # [  196.093836] gitaly[1389]: time="2026-06-29T10:46:24.067Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSGZ651EC2T4H8V643V3V diskcache=d5e80dfc-4d06-437d-be7c-c0fec9528d8e grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-29T10:46:33.181" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T10:46:23.971" pid=1389 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1826vm-test-run-nixbot-gitlab> gitlab # [  196.095796] gitaly[1389]: time="2026-06-29T10:46:24.069Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=72 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4854 command.oublock=8 command.real_time_ms=87 command.spawn_token_fork_ms=1 command.system_time_ms=39 command.user_time_ms=33 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSGZ651EC2T4H8V643V3V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-29T10:46:33.181" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T10:46:23.971" grpc.time_ms=97.422 pid=1389 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1827vm-test-run-nixbot-gitlab> gitlab # [  196.170777] gitaly[1389]: time="2026-06-29T10:46:24.144Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSGZ651EC2T4H8V643V3V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-29T10:46:34.003" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T10:46:24.144" grpc.time_ms=0.162 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1828vm-test-run-nixbot-gitlab> gitlab # [  196.190304] gitaly[1389]: time="2026-06-29T10:46:24.164Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=937 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSGZ651EC2T4H8V643V3V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-06-29T10:46:34.013" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T10:46:24.153" grpc.time_ms=10.055 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1829vm-test-run-nixbot-gitlab> gitlab # [  196.273080] gitaly[1389]: time="2026-06-29T10:46:24.246Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=338 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSGZ651EC2T4H8V643V3V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-06-29T10:46:54.041" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:24.241" grpc.time_ms=4.985 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1830vm-test-run-nixbot-gitlab> gitlab # [  196.311731] gitaly[1389]: time="2026-06-29T10:46:24.285Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSGZ651EC2T4H8V643V3V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-29T10:46:34.144" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T10:46:24.284" grpc.time_ms=0.246 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1831vm-test-run-nixbot-gitlab> gitlab # [  196.330559] gitaly[1389]: time="2026-06-29T10:46:24.296Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=343 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSGZ651EC2T4H8V643V3V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-06-29T10:46:34.151" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-06-29T10:46:24.291" grpc.time_ms=4.529 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1832vm-test-run-nixbot-gitlab> gitlab # [  196.380143] gitaly[1389]: time="2026-06-29T10:46:24.353Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSGZ651EC2T4H8V643V3V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T10:46:34.206" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T10:46:24.347" grpc.time_ms=6.435 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1833vm-test-run-nixbot-gitlab> gitlab # [  196.407107] gitaly[1389]: time="2026-06-29T10:46:24.380Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSGZ651EC2T4H8V643V3V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T10:46:34.231" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T10:46:24.371" grpc.time_ms=9.08 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1834vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for unit setup-gitlab.service, in 197.42 seconds)1835vm-test-run-nixbot-gitlab> gitlab: waiting for unit nixbot.service1836vm-test-run-nixbot-gitlab> gitlab # [  196.596626] gitaly[1389]: time="2026-06-29T10:46:24.570Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=959 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSGZ651EC2T4H8V643V3V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-29T10:46:54.354" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:24.554" grpc.time_ms=15.689 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1837vm-test-run-nixbot-gitlab> gitlab # [  196.812637] nixbot[3048]: {"ts": "2026-06-29T10:46:24.785Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1838vm-test-run-nixbot-gitlab> gitlab # [  196.831759] nixbot[3048]: {"ts": "2026-06-29T10:46:24.805Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1839vm-test-run-nixbot-gitlab> gitlab # [  196.844898] nixbot[3048]: {"ts": "2026-06-29T10:46:24.818Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1840vm-test-run-nixbot-gitlab> gitlab # [  196.850044] nixbot[3048]: {"ts": "2026-06-29T10:46:24.823Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1841vm-test-run-nixbot-gitlab> gitlab # [  196.854797] nixbot[3048]: {"ts": "2026-06-29T10:46:24.828Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1842vm-test-run-nixbot-gitlab> gitlab # [  196.859497] nixbot[3048]: {"ts": "2026-06-29T10:46:24.833Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1843vm-test-run-nixbot-gitlab> gitlab # [  196.866295] nixbot[3048]: {"ts": "2026-06-29T10:46:24.839Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1844vm-test-run-nixbot-gitlab> gitlab # [  196.871865] nixbot[3048]: {"ts": "2026-06-29T10:46:24.845Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1845vm-test-run-nixbot-gitlab> gitlab # [  196.880190] nixbot[3048]: {"ts": "2026-06-29T10:46:24.853Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1846vm-test-run-nixbot-gitlab> gitlab # [  196.885860] nixbot[3048]: {"ts": "2026-06-29T10:46:24.859Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1847vm-test-run-nixbot-gitlab> gitlab # [  196.890327] nixbot[3048]: {"ts": "2026-06-29T10:46:24.863Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1848vm-test-run-nixbot-gitlab> gitlab # [  196.898426] nixbot[3048]: {"ts": "2026-06-29T10:46:24.872Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1849vm-test-run-nixbot-gitlab> gitlab # [  196.902276] nixbot[3048]: {"ts": "2026-06-29T10:46:24.875Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1850vm-test-run-nixbot-gitlab> gitlab # [  196.905289] nixbot[3048]: {"ts": "2026-06-29T10:46:24.879Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1851vm-test-run-nixbot-gitlab> gitlab # [  196.908330] nixbot[3048]: {"ts": "2026-06-29T10:46:24.882Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1852vm-test-run-nixbot-gitlab> gitlab # [  196.910336] nixbot[3048]: {"ts": "2026-06-29T10:46:24.884Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1853vm-test-run-nixbot-gitlab> gitlab # [  196.912229] nixbot[3048]: {"ts": "2026-06-29T10:46:24.885Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1854vm-test-run-nixbot-gitlab> gitlab # [  196.914212] nixbot[3048]: {"ts": "2026-06-29T10:46:24.887Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1855vm-test-run-nixbot-gitlab> gitlab # [  197.199223] nixbot[3048]: INFO:     Started server process [3048]1856vm-test-run-nixbot-gitlab> gitlab # [  197.201207] nixbot[3048]: INFO:     Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)1857vm-test-run-nixbot-gitlab> gitlab # [  197.204721] nixbot[3048]: INFO:     ::ffff:127.0.0.1:59794 - "GET /health HTTP/1.1" 200 OK1858vm-test-run-nixbot-gitlab> gitlab # [  197.209574] systemd[1]: Started nixbot CI service.1859vm-test-run-nixbot-gitlab> gitlab # [  197.211223] systemd[1]: Reached target Multi-User System.1860vm-test-run-nixbot-gitlab> gitlab # [  197.213121] systemd[1]: Startup finished in 1.665s (kernel) + 4.284s (initrd) + 3min 11.260s (userspace) = 3min 17.211s.1861vm-test-run-nixbot-gitlab> gitlab # [  197.394703] gitaly[1389]: time="2026-06-29T10:46:25.368Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=967 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSK4SHD42TRR3T74X649P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-29T10:46:55.157" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:25.358" grpc.time_ms=9.933 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1862vm-test-run-nixbot-gitlab> gitlab # [  197.449794] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:25 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 2451863vm-test-run-nixbot-gitlab> gitlab # [  197.450928] nixbot[3048]: {"ts": "2026-06-29T10:46:25.423Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1864vm-test-run-nixbot-gitlab> gitlab # [  197.464170] nixbot[3048]: {"ts": "2026-06-29T10:46:25.437Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1865vm-test-run-nixbot-gitlab> gitlab # [  197.519055] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:25 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 441866vm-test-run-nixbot-gitlab> gitlab # [  197.523189] nixbot[3048]: {"ts": "2026-06-29T10:46:25.493Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1867vm-test-run-nixbot-gitlab> gitlab # [  197.523933] nixbot[3048]: {"ts": "2026-06-29T10:46:25.494Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1868vm-test-run-nixbot-gitlab> gitlab # [  197.575137] gitaly[1389]: time="2026-06-29T10:46:25.548Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSGZ651EC2T4H8V643V3V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-06-29T10:46:35.006" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:25.546" grpc.time_ms=1.61 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1869vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for unit nixbot.service, in 1.06 seconds)1870vm-test-run-nixbot-gitlab> gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1871vm-test-run-nixbot-gitlab> gitlab # [  197.640909] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:25.614Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 f55b93e3be80c71b3412c6f12756acbfb61aa22b refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"28741fa07f143cdae31f3c99","created_at":"2026-06-29T10:46:23.574Z","trace_propagation_headers":{"sentry-trace":"0137635c5bf24646accde69c40ec75da-a73b6f95ed7645e7","baggage":"sentry-trace_id=0137635c5bf24646accde69c40ec75da,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9FSGZ651EC2T4H8V643V3V","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:23.578Z","job_size_bytes":185,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-28741fa07f143cdae31f3c99: done: 2.025317 sec","job_status":"done","queue_duration_s":0.010137,"scheduling_latency_s":0.010171,"gitaly_calls":13,"gitaly_duration_s":0.456074,"redis_calls":209,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.26987500000000003,"redis_read_bytes":1475,"redis_write_bytes":20802,"redis_cache_calls":14,"redis_cache_duration_s":0.022233,"redis_cache_read_bytes":16,"redis_cache_write_bytes":884,"redis_feature_flag_calls":42,"redis_feature_flag_duration_s":0.029265,"redis_feature_flag_read_bytes":1058,"redis_feature_flag_write_bytes":5352,"redis_queues_calls":8,"redis_queues_duration_s":0.002162,"redis_queues_read_bytes":8,"redis_queues_write_bytes":4935,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.005004,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1259,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.189926,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.021285,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":54,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":54,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.132,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":54,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.881,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.902,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.132,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.507952,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001137295,"exclusive_lock_hold_duration_s":0.096880749,"duration_s":2.025317,"completed_at":"2026-06-29T10:46:25.614Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.175318,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1872vm-test-run-nixbot-gitlab> gitlab # [  197.723242] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:25.696Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"8e5672cb98f7841802ee591d","created_at":"2026-06-29T10:46:23.862Z","trace_propagation_headers":{"sentry-trace":"0137635c5bf24646accde69c40ec75da-113e33d7774942db","baggage":"sentry-trace_id=0137635c5bf24646accde69c40ec75da,sentry-release=Unknown"},"correlation_id":"01KW9FSGZ651EC2T4H8V643V3V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:23.871Z","job_size_bytes":46,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-8e5672cb98f7841802ee591d: start","job_status":"start","queue_duration_s":1.824714,"scheduling_latency_s":1.824751}1873vm-test-run-nixbot-gitlab> gitlab # [  197.728345] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:25 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 868 "" "python-httpx/0.28.1" 2011874vm-test-run-nixbot-gitlab> gitlab # [  197.729144] nixbot[3048]: {"ts": "2026-06-29T10:46:25.702Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""}1875vm-test-run-nixbot-gitlab> gitlab # [  197.763114] gitaly[1389]: time="2026-06-29T10:46:25.736Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSGZ651EC2T4H8V643V3V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-06-29T16:46:25.733" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T10:46:25.733" grpc.time_ms=2.156 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1876vm-test-run-nixbot-gitlab> gitlab # [  197.776336] gitaly[1389]: time="2026-06-29T10:46:25.748Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=648 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSGZ651EC2T4H8V643V3V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-06-29T10:46:55.541" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:25.741" grpc.time_ms=6.612 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1877vm-test-run-nixbot-gitlab> gitlab # [  197.803854] nixbot[3048]: INFO:     ::ffff:127.0.0.1:59802 - "GET /health HTTP/1.1" 200 OK1878vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.17 seconds)1879vm-test-run-nixbot-gitlab> subtest: project discovered and webhook registered1880vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1881vm-test-run-nixbot-gitlab> gitlab # [  197.829478] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:25.802Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"8e5672cb98f7841802ee591d","created_at":"2026-06-29T10:46:23.862Z","trace_propagation_headers":{"sentry-trace":"0137635c5bf24646accde69c40ec75da-113e33d7774942db","baggage":"sentry-trace_id=0137635c5bf24646accde69c40ec75da,sentry-release=Unknown"},"correlation_id":"01KW9FSGZ651EC2T4H8V643V3V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:23.871Z","job_size_bytes":46,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-8e5672cb98f7841802ee591d: done: 0.105667 sec","job_status":"done","queue_duration_s":1.824714,"scheduling_latency_s":1.824751,"gitaly_calls":2,"gitaly_duration_s":0.039009,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.008111,"redis_read_bytes":68,"redis_write_bytes":4882,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.00161,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000893,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2111,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001907,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.003034,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":752,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000667,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.045298,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000351721,"duration_s":0.105667,"completed_at":"2026-06-29T10:46:25.802Z","load_balancing_strategy":"primary","db_duration_s":0.010444,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1882vm-test-run-nixbot-gitlab> gitlab # [  197.835270] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:25.805Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"cf549085fe8b2b5762189f01","created_at":"2026-06-29T10:46:23.882Z","trace_propagation_headers":{"sentry-trace":"0137635c5bf24646accde69c40ec75da-113e33d7774942db","baggage":"sentry-trace_id=0137635c5bf24646accde69c40ec75da,sentry-release=Unknown"},"correlation_id":"01KW9FSGZ651EC2T4H8V643V3V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:23.890Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-cf549085fe8b2b5762189f01: start","job_status":"start","queue_duration_s":1.914779,"scheduling_latency_s":1.914801}1883vm-test-run-nixbot-gitlab> gitlab # [  197.911570] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:25 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "curl/8.20.0" 831884vm-test-run-nixbot-gitlab> gitlab # [  197.925462] gitaly[1389]: time="2026-06-29T10:46:25.899Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSGZ651EC2T4H8V643V3V diskcache=5bb9dddf-5793-45c7-974f-e73699b97248 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-29T16:46:25.824" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:25.825" pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1885vm-test-run-nixbot-gitlab> gitlab # [  197.926949] gitaly[1389]: time="2026-06-29T10:46:25.899Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1280 command.oublock=0 command.real_time_ms=71 command.spawn_token_fork_ms=4 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSGZ651EC2T4H8V643V3V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-29T16:46:25.824" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:25.825" grpc.time_ms=73.684 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1886vm-test-run-nixbot-gitlab> gitlab # [  197.955478] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:25.928Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"cf549085fe8b2b5762189f01","created_at":"2026-06-29T10:46:23.882Z","trace_propagation_headers":{"sentry-trace":"0137635c5bf24646accde69c40ec75da-113e33d7774942db","baggage":"sentry-trace_id=0137635c5bf24646accde69c40ec75da,sentry-release=Unknown"},"correlation_id":"01KW9FSGZ651EC2T4H8V643V3V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:23.890Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-cf549085fe8b2b5762189f01: done: 0.123968 sec","job_status":"done","queue_duration_s":1.914779,"scheduling_latency_s":1.914801,"gitaly_calls":1,"gitaly_duration_s":0.076447,"redis_calls":15,"redis_duration_s":0.003777,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000437,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001523,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000701,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001116,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.008,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.014,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030629,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.103719047,"duration_s":0.123968,"completed_at":"2026-06-29T10:46:25.928Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012599,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1887vm-test-run-nixbot-gitlab> gitlab # [  198.025392] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:25.930Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"a2b1873e21e9d4a1ea86b1e6","created_at":"2026-06-29T10:46:24.137Z","trace_propagation_headers":{"sentry-trace":"0137635c5bf24646accde69c40ec75da-113e33d7774942db","baggage":"sentry-trace_id=0137635c5bf24646accde69c40ec75da,sentry-release=Unknown"},"correlation_id":"01KW9FSGZ651EC2T4H8V643V3V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:24.139Z","job_size_bytes":23,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-a2b1873e21e9d4a1ea86b1e6: start","job_status":"start","queue_duration_s":1.790857,"scheduling_latency_s":1.790884}1888vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.27 seconds)1889vm-test-run-nixbot-gitlab> [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-06-29T10:46:25.657Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': False, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]1890vm-test-run-nixbot-gitlab> (finished: subtest: project discovered and webhook registered, in 0.27 seconds)1891vm-test-run-nixbot-gitlab> subtest: push delivers the webhook1892vm-test-run-nixbot-gitlab> gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1893vm-test-run-nixbot-gitlab> gitlab # [  198.078111] gitaly[1389]: time="2026-06-29T10:46:26.051Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=647 command.oublock=0 command.real_time_ms=91 command.spawn_token_fork_ms=86 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSGZ651EC2T4H8V643V3V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-06-29T10:46:55.758" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:25.959" grpc.time_ms=92.588 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1894vm-test-run-nixbot-gitlab> gitlab # [  198.089504] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:26.062Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"a2b1873e21e9d4a1ea86b1e6","created_at":"2026-06-29T10:46:24.137Z","trace_propagation_headers":{"sentry-trace":"0137635c5bf24646accde69c40ec75da-113e33d7774942db","baggage":"sentry-trace_id=0137635c5bf24646accde69c40ec75da,sentry-release=Unknown"},"correlation_id":"01KW9FSGZ651EC2T4H8V643V3V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:24.139Z","job_size_bytes":23,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-a2b1873e21e9d4a1ea86b1e6: done: 0.132308 sec","job_status":"done","queue_duration_s":1.790857,"scheduling_latency_s":1.790884,"gitaly_calls":1,"gitaly_duration_s":0.096448,"redis_calls":16,"redis_duration_s":0.004818,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.000312,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001317,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.002314,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000875,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0256,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.132308,"completed_at":"2026-06-29T10:46:26.062Z","load_balancing_strategy":"primary","db_duration_s":0.004174,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1895vm-test-run-nixbot-gitlab> gitlab # [  198.100117] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:26.064Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"3d2b3669c8728470deb9c017","created_at":"2026-06-29T10:46:25.519Z","trace_propagation_headers":{"sentry-trace":"0137635c5bf24646accde69c40ec75da-113e33d7774942db","baggage":"sentry-trace_id=0137635c5bf24646accde69c40ec75da,sentry-release=Unknown"},"correlation_id":"01KW9FSGZ651EC2T4H8V643V3V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:49404ad3a732ca64d16af4fb1708591f15ce83dd9eb9f82afd4b9dcaf9aafc19","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:25.523Z","job_size_bytes":467,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-3d2b3669c8728470deb9c017: start","job_status":"start","queue_duration_s":0.540975,"scheduling_latency_s":0.541002}1896vm-test-run-nixbot-gitlab> gitlab # [  198.173548] gitaly[1389]: time="2026-06-29T10:46:26.104Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSGZ651EC2T4H8V643V3V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T10:46:56.002" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:26.102" grpc.time_ms=1.373 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1897vm-test-run-nixbot-gitlab> gitlab # [  198.176225] gitaly[1389]: time="2026-06-29T10:46:26.114Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=621 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSGZ651EC2T4H8V643V3V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-29T10:46:36.006" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T10:46:26.107" grpc.time_ms=6.941 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1898vm-test-run-nixbot-gitlab> gitlab # [  198.221588] gitaly[1389]: time="2026-06-29T10:46:26.195Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSGZ651EC2T4H8V643V3V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-29T10:46:56.093" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:26.194" grpc.time_ms=1.022 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1899vm-test-run-nixbot-gitlab> gitlab # [  198.229316] gitaly[1389]: time="2026-06-29T10:46:26.203Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=174 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSGZ651EC2T4H8V643V3V error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-06-29T10:46:56.099" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:26.199" grpc.time_ms=2.857 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1900vm-test-run-nixbot-gitlab> gitlab # [  198.305510] gitaly[1389]: time="2026-06-29T10:46:26.279Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=621 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSGZ651EC2T4H8V643V3V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T10:46:36.171" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T10:46:26.271" grpc.time_ms=7.289 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1901vm-test-run-nixbot-gitlab> gitlab # [  198.373158] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:26 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 411902vm-test-run-nixbot-gitlab> gitlab # [  198.416681] nixbot[3048]: {"ts": "2026-06-29T10:46:26.388Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1903vm-test-run-nixbot-gitlab> gitlab # [  198.417502] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:26 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 6801904vm-test-run-nixbot-gitlab> gitlab # [  198.469214] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:26 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 202 "" "git/2.54.0" 921905vm-test-run-nixbot-gitlab> gitlab # [  198.471476] gitaly[1389]: time="2026-06-29T10:46:26.444Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSM9C81142S1JDJMAZT2H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=246 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T10:46:26.430" grpc.time_ms=9.651 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1906vm-test-run-nixbot-gitlab> gitlab # Enumerating objects: 5, done.1907vm-test-run-nixbot-gitlab> gitlab # Counting objects:  20% (1/5)Counting objects:  40% (2/5)Counting objects:  60% (3/5)Counting objects:  80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1908vm-test-run-nixbot-gitlab> gitlab # Delta compression using up to 4 threads1909vm-test-run-nixbot-gitlab> gitlab # Compressing objects:  50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1910vm-test-run-nixbot-gitlab> gitlab # Writing objects:  33% (1/3)Writing objects:  66% (2/3)Writing objects: 100% (3/3)[  198.509776] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:26 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "python-httpx/0.28.1" 821911vm-test-run-nixbot-gitlab> gitlab # [  198.514081] nixbot[3048]: {"ts": "2026-06-29T10:46:26.487Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1912vm-test-run-nixbot-gitlab> gitlab # Writing objects: 100% (3/3), 274 bytes | 27.00 KiB/s, done.1913vm-test-run-nixbot-gitlab> gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1914vm-test-run-nixbot-gitlab> gitlab # [  198.577501] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:26 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 868 "" "python-httpx/0.28.1" 551915vm-test-run-nixbot-gitlab> gitlab # [  198.582737] nixbot[3048]: {"ts": "2026-06-29T10:46:26.555Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""}1916vm-test-run-nixbot-gitlab> gitlab # [  198.655599] gitaly[1389]: time="2026-06-29T10:46:26.629Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=6 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSMG7QKAV6M5D0M38BP6V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-06-29T10:46:56.417" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T10:46:26.617" grpc.time_ms=11.512 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1917vm-test-run-nixbot-gitlab> gitlab # [  198.692164] gitaly[1389]: time="2026-06-29T10:46:26.665Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSMG7QKAV6M5D0M38BP6V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T10:46:56.464" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:26.664" grpc.time_ms=1.035 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1918vm-test-run-nixbot-gitlab> gitlab # [  198.706791] gitaly[1389]: time="2026-06-29T10:46:26.680Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSMG7QKAV6M5D0M38BP6V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-06-29T10:46:36.230" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:26.671" grpc.time_ms=9.143 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1919vm-test-run-nixbot-gitlab> gitlab # [  198.708821] gitaly[1389]: time="2026-06-29T10:46:26.682Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSME0T0XEV112TTXY7CGP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-06-29T10:46:36.007" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=253 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:26.669" grpc.time_ms=12.741 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1920vm-test-run-nixbot-gitlab> gitlab # [  198.751343] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:26 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 731 "" "python-httpx/0.28.1" 1551921vm-test-run-nixbot-gitlab> gitlab # [  198.752746] nixbot[3048]: {"ts": "2026-06-29T10:46:26.724Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""}1922vm-test-run-nixbot-gitlab> gitlab # [  198.770055] gitaly[1389]: time="2026-06-29T10:46:26.742Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=329 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSME0T0XEV112TTXY7CGP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-06-29T10:46:56.032" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-06-29T10:46:26.732" grpc.time_ms=9.632 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1923vm-test-run-nixbot-gitlab> gitlab # [  198.784991] gitaly[1389]: time="2026-06-29T10:46:26.758Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KW9FSME0T0XEV112TTXY7CGP duration_ms=144ns method=POST pid=1389 status=200 url="http://unix/api/v4/internal/allowed"1924vm-test-run-nixbot-gitlab> gitlab # [  198.809559] gitaly[1389]: time="2026-06-29T10:46:26.779Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KW9FSME0T0XEV112TTXY7CGP duration_ms=18ns method=POST pid=1389 status=200 url="http://unix/api/v4/internal/pre_receive"1925vm-test-run-nixbot-gitlab> gitlab # [  198.810918] gitaly[1389]: time="2026-06-29T10:46:26.779Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9FSME0T0XEV112TTXY7CGP grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5319 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T10:46:26.613" grpc.time_ms=166.073 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1926vm-test-run-nixbot-gitlab> gitlab # [  198.840239] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:26 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 821927vm-test-run-nixbot-gitlab> gitlab # [  198.843207] nixbot[3048]: {"ts": "2026-06-29T10:46:26.813Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""}1928vm-test-run-nixbot-gitlab> gitlab # [  198.852710] nixbot[3048]: {"ts": "2026-06-29T10:46:26.824Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1929vm-test-run-nixbot-gitlab> gitlab # [  198.862279] gitaly[1389]: time="2026-06-29T10:46:26.835Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9FSME0T0XEV112TTXY7CGP grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4789 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T10:46:26.835" grpc.time_ms=0.488 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1930vm-test-run-nixbot-gitlab> gitlab # [  198.905113] gitaly[1389]: time="2026-06-29T10:46:26.878Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9FSME0T0XEV112TTXY7CGP grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4788 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T10:46:26.878" grpc.time_ms=0.36 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1931vm-test-run-nixbot-gitlab> gitlab # [  198.948740] gitaly[1389]: time="2026-06-29T10:46:26.922Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9FSME0T0XEV112TTXY7CGP grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4790 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T10:46:26.922" grpc.time_ms=0.322 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1932vm-test-run-nixbot-gitlab> gitlab # [  199.004546] gitaly[1389]: time="2026-06-29T10:46:26.977Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSMSRJCYTT9YZZFDMPJRC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T10:46:26.966" grpc.time_ms=10.915 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc1933vm-test-run-nixbot-gitlab> gitlab # [  199.026052] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:26 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 1061934vm-test-run-nixbot-gitlab> gitlab # [  199.029996] gitaly[1389]: time="2026-06-29T10:46:27.000Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=1 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=964 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSGZ651EC2T4H8V643V3V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-29T10:46:56.873" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:26.977" grpc.time_ms=18.057 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1935vm-test-run-nixbot-gitlab> gitlab # [  199.055961] gitaly[1389]: time="2026-06-29T10:46:27.023Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KW9FSME0T0XEV112TTXY7CGP duration_ms=42ns method=POST pid=1389 status=200 url="http://unix/api/v4/internal/post_receive"1936vm-test-run-nixbot-gitlab> gitlab # [  199.061091] gitaly[1389]: time="2026-06-29T10:46:27.023Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9FSME0T0XEV112TTXY7CGP grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4813 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T10:46:26.973" grpc.time_ms=50.008 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1937vm-test-run-nixbot-gitlab> gitlab # [  199.079795] gitaly[1389]: time="2026-06-29T10:46:27.028Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=20 command.spawn_token_fork_ms=19 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSGZ651EC2T4H8V643V3V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-06-29T10:46:56.900" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:27.000" grpc.time_ms=28.203 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1938vm-test-run-nixbot-gitlab> gitlab # [  199.101767] gitaly[1389]: time="2026-06-29T10:46:27.046Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KW9FSME0T0XEV112TTXY7CGP diskcache=052cd92c-c309-4ed7-8e74-0ddc6378b60d grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T10:46:26.558" pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1939vm-test-run-nixbot-gitlab> gitlab # To http://gitlab/root/test-flake.git1940vm-test-run-nixbot-gitlab> gitlab #    f55b93e..ee1f362  master -> master1941vm-test-run-nixbot-gitlab> gitlab # [  199.119928] gitlab-workhorse[1291]: time="2026-06-29T10:46:27Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521942vm-test-run-nixbot-gitlab> gitlab # [  199.123264] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:27 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 5561943vm-test-run-nixbot-gitlab> gitlab # [  199.126653] gitaly[1389]: time="2026-06-29T10:46:27.052Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=173 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=872448 command.minflt=12851 command.oublock=32 command.real_time_ms=474 command.spawn_token_fork_ms=0 command.system_time_ms=134 command.user_time_ms=39 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSME0T0XEV112TTXY7CGP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=606 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=58 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T10:46:26.558" grpc.time_ms=488.602 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1944vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.08 seconds)1945vm-test-run-nixbot-gitlab> gitlab: must succeed: git -C /tmp/test-flake rev-parse master1946vm-test-run-nixbot-gitlab> gitlab # [  199.148726] sidekiq-cluster[2114]: {"severity":"WARN","time":"2026-06-29T10:46:27.102Z","class":"CreatePipelineWorker","correlation_id":"01KW9FSGZ651EC2T4H8V643V3V","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"f55b93e3be80c71b3412c6f12756acbfb61aa22b","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"f55b93e3be80c71b3412c6f12756acbfb61aa22b","ref":"refs/heads/master"},"retry":0}1947vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.04 seconds)1948vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee1f362830143952a51bf976a49d3d77debbfb0c'1949vm-test-run-nixbot-gitlab> gitlab # [  199.160371] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:27.108Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"3d2b3669c8728470deb9c017","created_at":"2026-06-29T10:46:25.519Z","trace_propagation_headers":{"sentry-trace":"0137635c5bf24646accde69c40ec75da-113e33d7774942db","baggage":"sentry-trace_id=0137635c5bf24646accde69c40ec75da,sentry-release=Unknown"},"correlation_id":"01KW9FSGZ651EC2T4H8V643V3V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:49404ad3a732ca64d16af4fb1708591f15ce83dd9eb9f82afd4b9dcaf9aafc19","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:25.523Z","job_size_bytes":467,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-3d2b3669c8728470deb9c017: done: 1.043797 sec","job_status":"done","queue_duration_s":0.540975,"scheduling_latency_s":0.541002,"gitaly_calls":7,"gitaly_duration_s":0.09229,"redis_calls":53,"redis_duration_s":0.010262,"redis_read_bytes":171,"redis_write_bytes":4427,"redis_cache_calls":4,"redis_cache_duration_s":0.001011,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.004542,"redis_feature_flag_read_bytes":18,"redis_feature_flag_write_bytes":2433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001336,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002368,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001005,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":40,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":40,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.073,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.064,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.669777,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000350324,"duration_s":1.043797,"completed_at":"2026-06-29T10:46:27.108Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.128183,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1950vm-test-run-nixbot-gitlab> gitlab # [  199.239048] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:27.109Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"49cb471a46edc15d03ac4237","created_at":"2026-06-29T10:46:25.784Z","trace_propagation_headers":{"sentry-trace":"0137635c5bf24646accde69c40ec75da-0e88fed655bf4390","baggage":"sentry-trace_id=0137635c5bf24646accde69c40ec75da,sentry-release=Unknown"},"correlation_id":"01KW9FSGZ651EC2T4H8V643V3V","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:25.789Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-49cb471a46edc15d03ac4237: start","job_status":"start","queue_duration_s":1.319984,"scheduling_latency_s":1.320006}1951vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee1f362830143952a51bf976a49d3d77debbfb0c', in 0.10 seconds)1952vm-test-run-nixbot-gitlab> []1953vm-test-run-nixbot-gitlab> gitlab # [  199.264866] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:27.154Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"49cb471a46edc15d03ac4237","created_at":"2026-06-29T10:46:25.784Z","trace_propagation_headers":{"sentry-trace":"0137635c5bf24646accde69c40ec75da-0e88fed655bf4390","baggage":"sentry-trace_id=0137635c5bf24646accde69c40ec75da,sentry-release=Unknown"},"correlation_id":"01KW9FSGZ651EC2T4H8V643V3V","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:25.789Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-49cb471a46edc15d03ac4237: done: 0.044917 sec","job_status":"done","queue_duration_s":1.319984,"scheduling_latency_s":1.320006,"redis_calls":19,"redis_duration_s":0.007186,"redis_read_bytes":21,"redis_write_bytes":4290,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000916,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000598,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2261,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.003708,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001964,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02345,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.008101588,"duration_s":0.044917,"completed_at":"2026-06-29T10:46:27.154Z","load_balancing_strategy":"primary","db_duration_s":0.011168,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1954vm-test-run-nixbot-gitlab> gitlab # [  199.337104] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:27.156Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","f55b93e3be80c71b3412c6f12756acbfb61aa22b ee1f362830143952a51bf976a49d3d77debbfb0c refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"9b0f84b7ec049f113b76127a","created_at":"2026-06-29T10:46:27.004Z","trace_propagation_headers":{"sentry-trace":"c45d966b5d6a4765ab4afca07a496def-7378dde5fc014def","baggage":"sentry-trace_id=c45d966b5d6a4765ab4afca07a496def,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9FSME0T0XEV112TTXY7CGP","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:27.010Z","job_size_bytes":185,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-9b0f84b7ec049f113b76127a: start","job_status":"start","queue_duration_s":0.146043,"scheduling_latency_s":0.146063}1955vm-test-run-nixbot-gitlab> gitlab # [  199.338178] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:27.166Z","message":"changes: \"f55b93e3be80c71b3412c6f12756acbfb61aa22b ee1f362830143952a51bf976a49d3d77debbfb0c refs/heads/master\\n\""}1956vm-test-run-nixbot-gitlab> gitlab # [  199.338759] nixbot[3048]: INFO:     ::ffff:127.0.0.1:59816 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee1f362830143952a51bf976a49d3d77debbfb0c HTTP/1.1" 200 OK1957vm-test-run-nixbot-gitlab> gitlab # [  199.340368] gitlab-workhorse[1291]: time="2026-06-29T10:46:27Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1491958vm-test-run-nixbot-gitlab> gitlab # [  199.341879] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:27 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 149 "" "git/2.54.0" 1471959vm-test-run-nixbot-gitlab> gitlab # [  199.343067] gitlab-workhorse[1291]: time="2026-06-29T10:46:27Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=5921960vm-test-run-nixbot-gitlab> gitlab # [  199.343905] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:27 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 592 "" "git/2.54.0" 821961vm-test-run-nixbot-gitlab> gitlab # [  199.344824] gitaly[1389]: time="2026-06-29T10:46:27.147Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSMXRBQ2JVHNHWB2PS44K grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T10:46:27.138" pid=1389 remote_ip="2001:db8:1::1" request_sha=083c43590fffa527704b32b8836424e8ed055017 response_bytes=149 span.kind=server system=grpc1962vm-test-run-nixbot-gitlab> gitlab # [  199.346044] gitaly[1389]: time="2026-06-29T10:46:27.147Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSMXRBQ2JVHNHWB2PS44K grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T10:46:27.138" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc1963vm-test-run-nixbot-gitlab> gitlab # [  199.350047] gitaly[1389]: time="2026-06-29T10:46:27.147Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSMXRBQ2JVHNHWB2PS44K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T10:46:27.138" grpc.time_ms=8.431 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc1964vm-test-run-nixbot-gitlab> gitlab # [  199.373456] gitaly[1389]: time="2026-06-29T10:46:27.189Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSME0T0XEV112TTXY7CGP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T10:46:37.081" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T10:46:27.182" grpc.time_ms=7.366 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1965vm-test-run-nixbot-gitlab> gitlab # [  199.443297] gitaly[1389]: time="2026-06-29T10:46:27.202Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSME0T0XEV112TTXY7CGP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-29T10:46:37.093" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T10:46:27.193" grpc.time_ms=8.431 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1966vm-test-run-nixbot-gitlab> gitlab # [  199.444885] gitaly[1389]: time="2026-06-29T10:46:27.231Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSN2FZ7SXHW5HAVHP140A grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T10:46:27.211" pid=1389 remote_ip="2001:db8:1::1" request_sha=e54f4460815a6e8d11dbcca554974148dca242f2 response_bytes=592 span.kind=server system=grpc1967vm-test-run-nixbot-gitlab> gitlab # [  199.445779] gitaly[1389]: time="2026-06-29T10:46:27.231Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1203 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSN2FZ7SXHW5HAVHP140A grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T10:46:27.211" negotiation.command=fetch negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc1968vm-test-run-nixbot-gitlab> gitlab # [  199.516625] gitaly[1389]: time="2026-06-29T10:46:27.232Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1203 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSN2FZ7SXHW5HAVHP140A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=9 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T10:46:27.211" grpc.time_ms=20.529 negotiation.command=fetch negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc1969vm-test-run-nixbot-gitlab> gitlab # [  199.517930] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:27.462Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","f55b93e3be80c71b3412c6f12756acbfb61aa22b ee1f362830143952a51bf976a49d3d77debbfb0c refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"9b0f84b7ec049f113b76127a","created_at":"2026-06-29T10:46:27.004Z","trace_propagation_headers":{"sentry-trace":"c45d966b5d6a4765ab4afca07a496def-7378dde5fc014def","baggage":"sentry-trace_id=c45d966b5d6a4765ab4afca07a496def,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9FSME0T0XEV112TTXY7CGP","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-29T10:46:27.010Z","job_size_bytes":185,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-9b0f84b7ec049f113b76127a: done: 0.306443 sec","job_status":"done","queue_duration_s":0.146043,"scheduling_latency_s":0.146063,"gitaly_calls":7,"gitaly_duration_s":0.096385,"redis_calls":112,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.032583,"redis_read_bytes":350,"redis_write_bytes":13070,"redis_cache_calls":9,"redis_cache_duration_s":0.008051,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000452,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":323,"redis_queues_calls":8,"redis_queues_duration_s":0.002155,"redis_queues_read_bytes":8,"redis_queues_write_bytes":6770,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.004087,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1182,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.009898,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.00794,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":29,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":29,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":29,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.018,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.018,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.15683,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000526882,"exclusive_lock_hold_duration_s":0.017587431,"duration_s":0.306443,"completed_at":"2026-06-29T10:46:27.462Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.02182,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1970vm-test-run-nixbot-gitlab> gitlab # [  199.605097] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:27.464Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"592176e805424d37e9dd069d","created_at":"2026-06-29T10:46:27.141Z","trace_propagation_headers":{"sentry-trace":"0137635c5bf24646accde69c40ec75da-2576c722a7674824","baggage":"sentry-trace_id=0137635c5bf24646accde69c40ec75da,sentry-release=Unknown"},"correlation_id":"01KW9FSGZ651EC2T4H8V643V3V","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:27.147Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-592176e805424d37e9dd069d: start","job_status":"start","queue_duration_s":0.316369,"scheduling_latency_s":0.316394}1971vm-test-run-nixbot-gitlab> gitlab # [  199.608267] gitaly[1389]: time="2026-06-29T10:46:27.244Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=20 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSME0T0XEV112TTXY7CGP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-06-29T10:46:37.122" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T10:46:27.222" grpc.time_ms=21.68 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1972vm-test-run-nixbot-gitlab> gitlab # [  199.609576] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:27.526Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"592176e805424d37e9dd069d","created_at":"2026-06-29T10:46:27.141Z","trace_propagation_headers":{"sentry-trace":"0137635c5bf24646accde69c40ec75da-2576c722a7674824","baggage":"sentry-trace_id=0137635c5bf24646accde69c40ec75da,sentry-release=Unknown"},"correlation_id":"01KW9FSGZ651EC2T4H8V643V3V","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:27.147Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-592176e805424d37e9dd069d: done: 0.062842 sec","job_status":"done","queue_duration_s":0.316369,"scheduling_latency_s":0.316394,"redis_calls":11,"redis_duration_s":0.002861,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000527,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001774,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00056,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042748,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000354514,"duration_s":0.062842,"completed_at":"2026-06-29T10:46:27.526Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014647,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1973vm-test-run-nixbot-gitlab> gitlab # [  199.686797] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:27.528Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"af38cb898486afed8ec9c363","created_at":"2026-06-29T10:46:27.204Z","trace_propagation_headers":{"sentry-trace":"c45d966b5d6a4765ab4afca07a496def-0f7e413fd9694b0d","baggage":"sentry-trace_id=c45d966b5d6a4765ab4afca07a496def,sentry-release=Unknown"},"correlation_id":"01KW9FSME0T0XEV112TTXY7CGP","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:27.207Z","job_size_bytes":46,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-af38cb898486afed8ec9c363: start","job_status":"start","queue_duration_s":0.321207,"scheduling_latency_s":0.321228}1974vm-test-run-nixbot-gitlab> gitlab # [  199.759181] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:27.544Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"af38cb898486afed8ec9c363","created_at":"2026-06-29T10:46:27.204Z","trace_propagation_headers":{"sentry-trace":"c45d966b5d6a4765ab4afca07a496def-0f7e413fd9694b0d","baggage":"sentry-trace_id=c45d966b5d6a4765ab4afca07a496def,sentry-release=Unknown"},"correlation_id":"01KW9FSME0T0XEV112TTXY7CGP","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:27.207Z","job_size_bytes":46,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-af38cb898486afed8ec9c363: done: 0.016493 sec","job_status":"done","queue_duration_s":0.321207,"scheduling_latency_s":0.321228,"redis_calls":9,"redis_duration_s":0.002241,"redis_read_bytes":23,"redis_write_bytes":782,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00135,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000259,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000632,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":304,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010429,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016493,"completed_at":"2026-06-29T10:46:27.544Z","load_balancing_strategy":"primary","db_duration_s":0.001283,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1975vm-test-run-nixbot-gitlab> gitlab # [  199.828991] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:27.546Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"b35d54387b63c94bf26676e5","created_at":"2026-06-29T10:46:27.211Z","trace_propagation_headers":{"sentry-trace":"c45d966b5d6a4765ab4afca07a496def-0f7e413fd9694b0d","baggage":"sentry-trace_id=c45d966b5d6a4765ab4afca07a496def,sentry-release=Unknown"},"correlation_id":"01KW9FSME0T0XEV112TTXY7CGP","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:27.214Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-b35d54387b63c94bf26676e5: start","job_status":"start","queue_duration_s":0.331716,"scheduling_latency_s":0.331737}1976vm-test-run-nixbot-gitlab> gitlab # [  199.830223] gitaly[1389]: time="2026-06-29T10:46:27.281Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=654 command.oublock=0 command.real_time_ms=29 command.spawn_token_fork_ms=26 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSME0T0XEV112TTXY7CGP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-06-29T10:46:57.148" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:27.248" grpc.time_ms=32.805 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1977vm-test-run-nixbot-gitlab> gitlab # [  199.831578] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:27.606Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"b35d54387b63c94bf26676e5","created_at":"2026-06-29T10:46:27.211Z","trace_propagation_headers":{"sentry-trace":"c45d966b5d6a4765ab4afca07a496def-0f7e413fd9694b0d","baggage":"sentry-trace_id=c45d966b5d6a4765ab4afca07a496def,sentry-release=Unknown"},"correlation_id":"01KW9FSME0T0XEV112TTXY7CGP","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:27.214Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-b35d54387b63c94bf26676e5: done: 0.060237 sec","job_status":"done","queue_duration_s":0.331716,"scheduling_latency_s":0.331737,"gitaly_calls":1,"gitaly_duration_s":0.035477,"redis_calls":12,"redis_duration_s":0.00292,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001219,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000654,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001047,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017718,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.045000133,"duration_s":0.060237,"completed_at":"2026-06-29T10:46:27.606Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003421,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1978vm-test-run-nixbot-gitlab> gitlab # [  199.925058] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:27.608Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"425d5d956939fbadeeac752c","created_at":"2026-06-29T10:46:27.348Z","trace_propagation_headers":{"sentry-trace":"c45d966b5d6a4765ab4afca07a496def-0f7e413fd9694b0d","baggage":"sentry-trace_id=c45d966b5d6a4765ab4afca07a496def,sentry-release=Unknown"},"correlation_id":"01KW9FSME0T0XEV112TTXY7CGP","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2861125560f2a574b5aac4541dfc028b1de87255b6108f04e45d2feb24ad1767","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:27.350Z","job_size_bytes":467,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-425d5d956939fbadeeac752c: start","job_status":"start","queue_duration_s":0.257285,"scheduling_latency_s":0.257307}1979vm-test-run-nixbot-gitlab> gitlab # [  199.928280] gitaly[1389]: time="2026-06-29T10:46:27.294Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=346 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSME0T0XEV112TTXY7CGP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-06-29T10:46:37.189" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-06-29T10:46:27.290" grpc.time_ms=3.869 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1980vm-test-run-nixbot-gitlab> gitlab # [  199.998542] gitaly[1389]: time="2026-06-29T10:46:27.305Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSME0T0XEV112TTXY7CGP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T10:46:37.199" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T10:46:27.300" grpc.time_ms=5.219 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1981vm-test-run-nixbot-gitlab> gitlab # [  200.000265] gitaly[1389]: time="2026-06-29T10:46:27.447Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSME0T0XEV112TTXY7CGP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-06-29T10:46:37.005" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:27.445" grpc.time_ms=2.222 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1982vm-test-run-nixbot-gitlab> gitlab # [  200.001296] gitaly[1389]: time="2026-06-29T10:46:27.594Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSME0T0XEV112TTXY7CGP diskcache=51e0a050-918c-4be2-887f-b2e44e43f133 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-29T16:46:27.560" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:27.561" pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1983vm-test-run-nixbot-gitlab> gitlab # [  200.002228] gitaly[1389]: time="2026-06-29T10:46:27.594Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=5 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1644 command.oublock=0 command.real_time_ms=33 command.spawn_token_fork_ms=6 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSME0T0XEV112TTXY7CGP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-29T16:46:27.560" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:27.561" grpc.time_ms=32.879 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1984vm-test-run-nixbot-gitlab> gitlab # [  200.025804] gitaly[1389]: time="2026-06-29T10:46:27.627Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSME0T0XEV112TTXY7CGP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T10:46:57.526" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:27.626" grpc.time_ms=0.711 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1985vm-test-run-nixbot-gitlab> gitlab # [  200.108102] gitaly[1389]: time="2026-06-29T10:46:27.636Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSME0T0XEV112TTXY7CGP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-29T10:46:37.188" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T10:46:27.629" grpc.time_ms=7.334 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1986vm-test-run-nixbot-gitlab> gitlab # [  200.109336] gitaly[1389]: time="2026-06-29T10:46:27.659Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSME0T0XEV112TTXY7CGP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-29T10:46:57.557" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:27.658" grpc.time_ms=1.114 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1987vm-test-run-nixbot-gitlab> gitlab # [  200.110193] gitaly[1389]: time="2026-06-29T10:46:27.666Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=297 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSME0T0XEV112TTXY7CGP error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-06-29T10:46:57.561" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:27.662" grpc.time_ms=4.356 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1988vm-test-run-nixbot-gitlab> gitlab # [  200.111891] gitaly[1389]: time="2026-06-29T10:46:27.789Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSME0T0XEV112TTXY7CGP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T10:46:37.240" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T10:46:27.781" grpc.time_ms=7.422 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1989vm-test-run-nixbot-gitlab> gitlab # [  200.251761] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:28 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 461990vm-test-run-nixbot-gitlab> gitlab # [  200.253219] gitaly[1389]: time="2026-06-29T10:46:28.224Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSP2JN0FP7PNYHE0CBMZK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T10:46:28.216" grpc.time_ms=7.428 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc1991vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee1f362830143952a51bf976a49d3d77debbfb0c'1992vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee1f362830143952a51bf976a49d3d77debbfb0c', in 0.04 seconds)1993vm-test-run-nixbot-gitlab> []1994vm-test-run-nixbot-gitlab> gitlab # [  200.330513] gitaly[1389]: time="2026-06-29T10:46:28.303Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSP4TF980QSHFP560YRKV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T10:46:28.295" pid=1389 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc1995vm-test-run-nixbot-gitlab> gitlab # [  200.331947] gitaly[1389]: time="2026-06-29T10:46:28.303Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=3 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSP4TF980QSHFP560YRKV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T10:46:28.295" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc1996vm-test-run-nixbot-gitlab> gitlab # [  200.335637] gitaly[1389]: time="2026-06-29T10:46:28.304Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=3 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSP4TF980QSHFP560YRKV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T10:46:28.295" grpc.time_ms=7.766 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc1997vm-test-run-nixbot-gitlab> gitlab # [  200.337129] gitlab-workhorse[1291]: time="2026-06-29T10:46:28Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671998vm-test-run-nixbot-gitlab> gitlab # [  200.337910] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:28 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 571999vm-test-run-nixbot-gitlab> gitlab # [  200.339403] nixbot[3048]: INFO:     ::ffff:127.0.0.1:59826 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee1f362830143952a51bf976a49d3d77debbfb0c HTTP/1.1" 200 OK2000vm-test-run-nixbot-gitlab> gitlab # [  200.432173] gitaly[1389]: time="2026-06-29T10:46:28.405Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=966 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSME0T0XEV112TTXY7CGP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-29T10:46:58.295" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:28.395" grpc.time_ms=10.103 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2001vm-test-run-nixbot-gitlab> gitlab # [  200.453853] gitaly[1389]: time="2026-06-29T10:46:28.427Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSME0T0XEV112TTXY7CGP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-06-29T10:46:58.318" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:28.418" grpc.time_ms=9.122 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2002vm-test-run-nixbot-gitlab> gitlab # [  200.464296] sidekiq-cluster[2114]: {"severity":"WARN","time":"2026-06-29T10:46:28.437Z","class":"CreatePipelineWorker","correlation_id":"01KW9FSME0T0XEV112TTXY7CGP","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"f55b93e3be80c71b3412c6f12756acbfb61aa22b","after":"ee1f362830143952a51bf976a49d3d77debbfb0c","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"ee1f362830143952a51bf976a49d3d77debbfb0c","ref":"refs/heads/master"},"retry":0}2003vm-test-run-nixbot-gitlab> gitlab # [  200.468529] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:28.441Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"425d5d956939fbadeeac752c","created_at":"2026-06-29T10:46:27.348Z","trace_propagation_headers":{"sentry-trace":"c45d966b5d6a4765ab4afca07a496def-0f7e413fd9694b0d","baggage":"sentry-trace_id=c45d966b5d6a4765ab4afca07a496def,sentry-release=Unknown"},"correlation_id":"01KW9FSME0T0XEV112TTXY7CGP","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2861125560f2a574b5aac4541dfc028b1de87255b6108f04e45d2feb24ad1767","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:27.350Z","job_size_bytes":467,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-425d5d956939fbadeeac752c: done: 0.83406 sec","job_status":"done","queue_duration_s":0.257285,"scheduling_latency_s":0.257307,"gitaly_calls":7,"gitaly_duration_s":0.146804,"redis_calls":33,"redis_duration_s":0.005792,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000834,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001279,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.00261,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001069,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":30,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.114,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":26,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.111,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.616333,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.83406,"completed_at":"2026-06-29T10:46:28.441Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.113731,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2004vm-test-run-nixbot-gitlab> gitlab # [  200.557426] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:28.443Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"46427c60-1ff1-4cc0-b10d-f3f331af2737\"}"],"class":"WebHookWorker","jid":"ad527068e2535062ceafbeb9","created_at":"2026-06-29T10:46:27.432Z","trace_propagation_headers":{"sentry-trace":"c45d966b5d6a4765ab4afca07a496def-0f7e413fd9694b0d","baggage":"sentry-trace_id=c45d966b5d6a4765ab4afca07a496def,sentry-release=Unknown"},"correlation_id":"01KW9FSME0T0XEV112TTXY7CGP","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d1a197d6c72f45ed1ab0fd3f498151100ccf0717ec94179d268b2e3091c89a29","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:27.437Z","job_size_bytes":1749,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-ad527068e2535062ceafbeb9: start","job_status":"start","queue_duration_s":1.006142,"scheduling_latency_s":1.006166}2005vm-test-run-nixbot-gitlab> gitlab # [  200.558846] nixbot[3048]: INFO:     ::ffff:127.0.0.1:59834 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted2006vm-test-run-nixbot-gitlab> gitlab # [  200.577888] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:28.551Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"46427c60-1ff1-4cc0-b10d-f3f331af2737\"}"],"class":"WebHookWorker","jid":"ad527068e2535062ceafbeb9","created_at":"2026-06-29T10:46:27.432Z","trace_propagation_headers":{"sentry-trace":"c45d966b5d6a4765ab4afca07a496def-0f7e413fd9694b0d","baggage":"sentry-trace_id=c45d966b5d6a4765ab4afca07a496def,sentry-release=Unknown"},"correlation_id":"01KW9FSME0T0XEV112TTXY7CGP","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d1a197d6c72f45ed1ab0fd3f498151100ccf0717ec94179d268b2e3091c89a29","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:27.437Z","job_size_bytes":1749,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-ad527068e2535062ceafbeb9: done: 0.107471 sec","job_status":"done","queue_duration_s":1.006142,"scheduling_latency_s":1.006166,"redis_calls":19,"redis_duration_s":0.009721,"redis_read_bytes":31,"redis_write_bytes":5281,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000992,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000361,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3643,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.007105,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001263,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.008114719000019477,"cpu_s":0.078119,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000440559,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","duration_s":0.107471,"completed_at":"2026-06-29T10:46:28.550Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001691,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2007vm-test-run-nixbot-gitlab> gitlab # [  200.599304] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:28.557Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","ad527068e2535062ceafbeb9"],"class":"WebHooks::LogExecutionWorker","jid":"b28a12c61a5bac4f55faf877","created_at":"2026-06-29T10:46:28.528Z","trace_propagation_headers":{"sentry-trace":"c45d966b5d6a4765ab4afca07a496def-eca3bad08b3d4d9a","baggage":"sentry-trace_id=c45d966b5d6a4765ab4afca07a496def,sentry-release=Unknown"},"correlation_id":"01KW9FSME0T0XEV112TTXY7CGP","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:50be739e16b35d6ee75b24105ef500ea88a9d51b4c704c7d766fcb0e0a096aa1","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:28.538Z","job_size_bytes":2506,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-b28a12c61a5bac4f55faf877: start","job_status":"start","queue_duration_s":0.018631,"scheduling_latency_s":0.018659}2008vm-test-run-nixbot-gitlab> gitlab # [  200.631279] gitaly[1389]: time="2026-06-29T10:46:28.590Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSPE14Q40WSCY413SD2FF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T10:46:28.582" grpc.time_ms=7.354 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc2009vm-test-run-nixbot-gitlab> gitlab # [  200.649181] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:28 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 462010vm-test-run-nixbot-gitlab> gitlab # [  200.668510] gitaly[1389]: time="2026-06-29T10:46:28.642Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSPFH19QDZMVTEE21PNTX grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T10:46:28.628" pid=1389 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc2011vm-test-run-nixbot-gitlab> gitlab # [  200.673289] gitaly[1389]: time="2026-06-29T10:46:28.642Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSPFH19QDZMVTEE21PNTX grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T10:46:28.628" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc2012vm-test-run-nixbot-gitlab> gitlab # [  200.745330] gitlab-workhorse[1291]: time="2026-06-29T10:46:28Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672013vm-test-run-nixbot-gitlab> gitlab # [  200.748159] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:28 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 502014vm-test-run-nixbot-gitlab> gitlab # [  200.749210] nixbot[3048]: {"ts": "2026-06-29T10:46:28.698Z", "level": "info", "logger": "nixbot.canceller", "msg": "cancelling build", "build_id": 1, "reason": "superseded"}2015vm-test-run-nixbot-gitlab> gitlab # [  200.749767] gitaly[1389]: time="2026-06-29T10:46:28.642Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSPFH19QDZMVTEE21PNTX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-29T10:46:28.628" grpc.time_ms=14.179 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc2016vm-test-run-nixbot-gitlab> gitlab # [  200.753527] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:28.726Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","ad527068e2535062ceafbeb9"],"class":"WebHooks::LogExecutionWorker","jid":"b28a12c61a5bac4f55faf877","created_at":"2026-06-29T10:46:28.528Z","trace_propagation_headers":{"sentry-trace":"c45d966b5d6a4765ab4afca07a496def-eca3bad08b3d4d9a","baggage":"sentry-trace_id=c45d966b5d6a4765ab4afca07a496def,sentry-release=Unknown"},"correlation_id":"01KW9FSME0T0XEV112TTXY7CGP","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:50be739e16b35d6ee75b24105ef500ea88a9d51b4c704c7d766fcb0e0a096aa1","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:28.538Z","job_size_bytes":2506,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-b28a12c61a5bac4f55faf877: done: 0.169177 sec","job_status":"done","queue_duration_s":0.018631,"scheduling_latency_s":0.018659,"redis_calls":10,"redis_duration_s":0.009665,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004327,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004264,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001074,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.028,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.028,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.109564,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000450057,"duration_s":0.169177,"completed_at":"2026-06-29T10:46:28.726Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01143,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2017vm-test-run-nixbot-gitlab> gitlab # [  200.840511] gitaly[1389]: time="2026-06-29T10:46:28.745Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSPJZFY11X1NHHSMWWPMM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T10:46:58.043" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:28.743" grpc.time_ms=1.172 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2018vm-test-run-nixbot-gitlab> gitlab # [  200.895412] gitaly[1389]: time="2026-06-29T10:46:28.869Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=649 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSPJZFY11X1NHHSMWWPMM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-06-29T10:46:58.159" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T10:46:28.860" grpc.time_ms=8.75 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2019vm-test-run-nixbot-gitlab> gitlab # [  200.909968] gitaly[1389]: time="2026-06-29T10:46:28.883Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSPJZFY11X1NHHSMWWPMM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T10:46:38.217" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T10:46:28.877" grpc.time_ms=5.714 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2020vm-test-run-nixbot-gitlab> gitlab # [  200.954165] gitaly[1389]: time="2026-06-29T10:46:28.927Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSPJZFY11X1NHHSMWWPMM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-29T10:46:38.266" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T10:46:28.926" grpc.time_ms=0.321 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2021vm-test-run-nixbot-gitlab> gitlab # [  200.977114] gitaly[1389]: time="2026-06-29T10:46:28.950Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSP9MJSACFYFKR43D4BXJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T10:46:58.749" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:28.949" grpc.time_ms=0.619 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2022vm-test-run-nixbot-gitlab> gitlab # [  200.998504] gitaly[1389]: time="2026-06-29T10:46:28.960Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=620 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSPJZFY11X1NHHSMWWPMM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-29T10:46:38.002" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T10:46:28.953" grpc.time_ms=6.565 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2023vm-test-run-nixbot-gitlab> gitlab # [  201.020465] gitaly[1389]: time="2026-06-29T10:46:28.991Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSP9MJSACFYFKR43D4BXJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-06-29T10:46:58.784" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T10:46:28.984" grpc.time_ms=6.9 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2024vm-test-run-nixbot-gitlab> gitlab # [  201.032385] gitaly[1389]: time="2026-06-29T10:46:29.006Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSP9MJSACFYFKR43D4BXJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T10:46:38.259" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T10:46:28.999" grpc.time_ms=6.235 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2025vm-test-run-nixbot-gitlab> gitlab # [  201.081186] gitaly[1389]: time="2026-06-29T10:46:29.054Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9FSPJZFY11X1NHHSMWWPMM grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4157 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T10:46:29.054" grpc.time_ms=0.394 pid=1389 span.kind=server system=grpc2026vm-test-run-nixbot-gitlab> gitlab # [  201.113580] gitaly[1389]: time="2026-06-29T10:46:29.087Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSP9MJSACFYFKR43D4BXJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-29T10:46:39.006" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T10:46:29.086" grpc.time_ms=0.274 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2027vm-test-run-nixbot-gitlab> gitlab # [  201.131302] gitaly[1389]: time="2026-06-29T10:46:29.092Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9FSPJZFY11X1NHHSMWWPMM grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4159 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T10:46:29.091" grpc.time_ms=0.73 pid=1389 span.kind=server system=grpc2028vm-test-run-nixbot-gitlab> gitlab # [  201.132332] gitaly[1389]: time="2026-06-29T10:46:29.097Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSP9MJSACFYFKR43D4BXJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-29T10:46:39.011" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T10:46:29.091" grpc.time_ms=5.458 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2029vm-test-run-nixbot-gitlab> gitlab # [  201.141525] gitaly[1389]: time="2026-06-29T10:46:29.115Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSPJZFY11X1NHHSMWWPMM diskcache=8b5a699b-d32d-4acf-8f42-19a0c163704d grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-29T10:46:38.022" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T10:46:28.972" pid=1389 ref=refs/keep-around/ee1f362830143952a51bf976a49d3d77debbfb0c remote_ip="2001:db8:1::1" revision=ee1f362830143952a51bf976a49d3d77debbfb0c span.kind=server system=grpc username=root2030vm-test-run-nixbot-gitlab> gitlab # [  201.143201] gitaly[1389]: time="2026-06-29T10:46:29.115Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=116 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5373 command.oublock=8 command.real_time_ms=132 command.spawn_token_fork_ms=1 command.system_time_ms=90 command.user_time_ms=26 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSPJZFY11X1NHHSMWWPMM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-29T10:46:38.022" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T10:46:28.972" grpc.time_ms=142.663 pid=1389 ref=refs/keep-around/ee1f362830143952a51bf976a49d3d77debbfb0c remote_ip="2001:db8:1::1" revision=ee1f362830143952a51bf976a49d3d77debbfb0c span.kind=server system=grpc username=root2031vm-test-run-nixbot-gitlab> gitlab # [  201.215122] gitaly[1389]: time="2026-06-29T10:46:29.188Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSPJZFY11X1NHHSMWWPMM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T10:46:59.087" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:29.187" grpc.time_ms=0.573 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2032vm-test-run-nixbot-gitlab> gitlab # [  201.218670] gitaly[1389]: time="2026-06-29T10:46:29.192Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9FSP9MJSACFYFKR43D4BXJ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4157 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T10:46:29.191" grpc.time_ms=0.447 pid=1389 span.kind=server system=grpc2033vm-test-run-nixbot-gitlab> gitlab # [  201.233360] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:29.206Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"dc516b22dba5273b9e2f539f","created_at":"2026-06-29T10:46:29.196Z","trace_propagation_headers":{"sentry-trace":"cd84d747e9684b57bc46840900c8ea52-577f5a8d2f5742ab","baggage":"sentry-trace_id=cd84d747e9684b57bc46840900c8ea52,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9FSPJZFY11X1NHHSMWWPMM","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:29.204Z","job_size_bytes":65,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-dc516b22dba5273b9e2f539f: start","job_status":"start","queue_duration_s":0.002004,"scheduling_latency_s":0.002055}2034vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee1f362830143952a51bf976a49d3d77debbfb0c'2035vm-test-run-nixbot-gitlab> gitlab # [  201.295108] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:29.267Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"dc516b22dba5273b9e2f539f","created_at":"2026-06-29T10:46:29.196Z","trace_propagation_headers":{"sentry-trace":"cd84d747e9684b57bc46840900c8ea52-577f5a8d2f5742ab","baggage":"sentry-trace_id=cd84d747e9684b57bc46840900c8ea52,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9FSPJZFY11X1NHHSMWWPMM","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:29.204Z","job_size_bytes":65,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-dc516b22dba5273b9e2f539f: done: 0.061247 sec","job_status":"done","queue_duration_s":0.002004,"scheduling_latency_s":0.002055,"redis_calls":19,"redis_duration_s":0.009058,"redis_read_bytes":145,"redis_write_bytes":2300,"redis_feature_flag_calls":11,"redis_feature_flag_duration_s":0.006491,"redis_feature_flag_read_bytes":138,"redis_feature_flag_write_bytes":1482,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001938,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000629,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034622,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000424356,"duration_s":0.061247,"completed_at":"2026-06-29T10:46:29.267Z","load_balancing_strategy":"primary","db_duration_s":0.012457,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2036vm-test-run-nixbot-gitlab> gitlab # [  201.299840] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:29.273Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"00f72631f1f3f4566cf1c6bc","created_at":"2026-06-29T10:46:29.206Z","trace_propagation_headers":{"sentry-trace":"cd84d747e9684b57bc46840900c8ea52-577f5a8d2f5742ab","baggage":"sentry-trace_id=cd84d747e9684b57bc46840900c8ea52,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9FSPJZFY11X1NHHSMWWPMM","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:29.211Z","job_size_bytes":65,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-00f72631f1f3f4566cf1c6bc: start","job_status":"start","queue_duration_s":0.061425,"scheduling_latency_s":0.06145}2037vm-test-run-nixbot-gitlab> gitlab # [  201.398703] gitaly[1389]: time="2026-06-29T10:46:29.372Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9FSP9MJSACFYFKR43D4BXJ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4159 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T10:46:29.372" grpc.time_ms=0.38 pid=1389 span.kind=server system=grpc2038vm-test-run-nixbot-gitlab> gitlab # [  201.410269] nixbot[3048]: INFO:     ::ffff:127.0.0.1:59848 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee1f362830143952a51bf976a49d3d77debbfb0c HTTP/1.1" 200 OK2039vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee1f362830143952a51bf976a49d3d77debbfb0c', in 0.08 seconds)2040vm-test-run-nixbot-gitlab> [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': 'ee1f362830143952a51bf976a49d3d77debbfb0c', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-06-29T10:46:28.663083Z', 'started_at': None, 'finished_at': None}]2041vm-test-run-nixbot-gitlab> (finished: subtest: push delivers the webhook, in 3.34 seconds)2042vm-test-run-nixbot-gitlab> subtest: eval and build post commit statuses2043vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee1f362830143952a51bf976a49d3d77debbfb0c/statuses2044vm-test-run-nixbot-gitlab> gitlab # [  201.425870] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:29.399Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"00f72631f1f3f4566cf1c6bc","created_at":"2026-06-29T10:46:29.206Z","trace_propagation_headers":{"sentry-trace":"cd84d747e9684b57bc46840900c8ea52-577f5a8d2f5742ab","baggage":"sentry-trace_id=cd84d747e9684b57bc46840900c8ea52,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9FSPJZFY11X1NHHSMWWPMM","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:29.211Z","job_size_bytes":65,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-00f72631f1f3f4566cf1c6bc: done: 0.12569 sec","job_status":"done","queue_duration_s":0.061425,"scheduling_latency_s":0.06145,"redis_calls":15,"redis_duration_s":0.029741999999999998,"redis_read_bytes":142,"redis_write_bytes":1431,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.00073,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":431,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.027138,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001874,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036006,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000746743,"duration_s":0.12569,"completed_at":"2026-06-29T10:46:29.399Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016793,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2045vm-test-run-nixbot-gitlab> gitlab # [  201.431628] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:29 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/ee1f362830143952a51bf976a49d3d77debbfb0c HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 6992046vm-test-run-nixbot-gitlab> gitlab # [  201.433188] nixbot[3048]: {"ts": "2026-06-29T10:46:29.403Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/ee1f362830143952a51bf976a49d3d77debbfb0c \"HTTP/1.1 201 Created\""}2047vm-test-run-nixbot-gitlab> gitlab # [  201.437906] nixbot[3048]: {"ts": "2026-06-29T10:46:29.411Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-ee1f36283014-7d08fd78", "argv0": "bwrap"}2048vm-test-run-nixbot-gitlab> gitlab # [  201.509693] gitaly[1389]: time="2026-06-29T10:46:29.483Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSP9MJSACFYFKR43D4BXJ diskcache=22671754-888b-4478-9164-6ade8d5e96a2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-29T10:46:39.028" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T10:46:29.108" pid=1389 ref=refs/keep-around/f55b93e3be80c71b3412c6f12756acbfb61aa22b remote_ip="2001:db8:1::1" revision=f55b93e3be80c71b3412c6f12756acbfb61aa22b span.kind=server system=grpc username=root2049vm-test-run-nixbot-gitlab> gitlab # [  201.512199] gitaly[1389]: time="2026-06-29T10:46:29.485Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=223 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=856064 command.minflt=5056 command.oublock=8 command.real_time_ms=282 command.spawn_token_fork_ms=0 command.system_time_ms=205 command.user_time_ms=18 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSP9MJSACFYFKR43D4BXJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-29T10:46:39.028" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-29T10:46:29.108" grpc.time_ms=375.79 pid=1389 ref=refs/keep-around/f55b93e3be80c71b3412c6f12756acbfb61aa22b remote_ip="2001:db8:1::1" revision=f55b93e3be80c71b3412c6f12756acbfb61aa22b span.kind=server system=grpc username=root2050vm-test-run-nixbot-gitlab> gitlab # [  201.534473] gitaly[1389]: time="2026-06-29T10:46:29.508Z" level=info msg="finished unary call with code OK" catfile.duration_ms=18 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=18 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSP9MJSACFYFKR43D4BXJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T10:46:59.288" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:29.488" grpc.time_ms=19.197 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2051vm-test-run-nixbot-gitlab> gitlab # [  201.556818] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:29.529Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"3f3e307e17d3f15c31ec750a","created_at":"2026-06-29T10:46:29.379Z","trace_propagation_headers":{"sentry-trace":"cd84d747e9684b57bc46840900c8ea52-577f5a8d2f5742ab","baggage":"sentry-trace_id=cd84d747e9684b57bc46840900c8ea52,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9FSPJZFY11X1NHHSMWWPMM","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:29.386Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-3f3e307e17d3f15c31ec750a: start","job_status":"start","queue_duration_s":0.1433,"scheduling_latency_s":0.143335}2052vm-test-run-nixbot-gitlab> gitlab # [  201.636834] gitaly[1389]: time="2026-06-29T10:46:29.608Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSPJZFY11X1NHHSMWWPMM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-29T10:46:59.507" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:29.607" grpc.time_ms=0.904 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2053vm-test-run-nixbot-gitlab> gitlab # [  201.670366] systemd[1]: Started Nix Daemon.2054vm-test-run-nixbot-gitlab> gitlab # [  201.684107] gitaly[1389]: time="2026-06-29T10:46:29.656Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSPJZFY11X1NHHSMWWPMM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T10:46:59.554" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:29.655" grpc.time_ms=0.677 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2055vm-test-run-nixbot-gitlab> gitlab # [  201.706704] nix-daemon[3413]: accepted connection from pid 3412, user nixbot2056vm-test-run-nixbot-gitlab> gitlab # [  201.711938] gitaly[1389]: time="2026-06-29T10:46:29.685Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSQEJXRSASJJ0GS4SD53J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T10:46:59.584" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:29.684" grpc.time_ms=0.773 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2057vm-test-run-nixbot-gitlab> gitlab # [  201.715427] nix-daemon[3413]: accepted connection from pid 3411, user nixbot2058vm-test-run-nixbot-gitlab> gitlab # [  201.739975] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:29 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/f55b93e3be80c71b3412c6f12756acbfb61aa22b HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 13082059vm-test-run-nixbot-gitlab> gitlab # [  201.744135] nixbot[3048]: {"ts": "2026-06-29T10:46:29.715Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/f55b93e3be80c71b3412c6f12756acbfb61aa22b \"HTTP/1.1 201 Created\""}2060vm-test-run-nixbot-gitlab> gitlab # [  201.749868] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:29 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/ee1f362830143952a51bf976a49d3d77debbfb0c/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1362061vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee1f362830143952a51bf976a49d3d77debbfb0c/statuses, in 0.34 seconds)2062vm-test-run-nixbot-gitlab> [{'id': 1, 'sha': 'ee1f362830143952a51bf976a49d3d77debbfb0c', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-29T10:46:29.369Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2063vm-test-run-nixbot-gitlab> gitlab # [  201.758655] gitaly[1389]: time="2026-06-29T10:46:29.732Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSPJZFY11X1NHHSMWWPMM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T10:46:59.630" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:29.731" grpc.time_ms=0.791 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2064vm-test-run-nixbot-gitlab> gitlab # [  201.789257] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:29.762Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"3f3e307e17d3f15c31ec750a","created_at":"2026-06-29T10:46:29.379Z","trace_propagation_headers":{"sentry-trace":"cd84d747e9684b57bc46840900c8ea52-577f5a8d2f5742ab","baggage":"sentry-trace_id=cd84d747e9684b57bc46840900c8ea52,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9FSPJZFY11X1NHHSMWWPMM","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:29.386Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-3f3e307e17d3f15c31ec750a: done: 0.232894 sec","job_status":"done","queue_duration_s":0.1433,"scheduling_latency_s":0.143335,"gitaly_calls":3,"gitaly_duration_s":0.031206,"redis_calls":53,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.014163,"redis_read_bytes":235,"redis_write_bytes":7159,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001253,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.004838,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_feature_flag_calls":7,"redis_feature_flag_duration_s":0.001557,"redis_feature_flag_read_bytes":134,"redis_feature_flag_write_bytes":992,"redis_queues_calls":2,"redis_queues_duration_s":0.000285,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.002186,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":840,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.00061,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.001191,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00099,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":34,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":34,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.044,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":22,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.005,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.007,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.026,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.093368,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.206031492,"duration_s":0.232894,"completed_at":"2026-06-29T10:46:29.762Z","load_balancing_strategy":"primary","db_duration_s":0.060183,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2065vm-test-run-nixbot-gitlab> gitlab # [  201.864618] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:29.764Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"cc38d601c59bcd89374f015e","created_at":"2026-06-29T10:46:29.573Z","trace_propagation_headers":{"sentry-trace":"1df619fe0f414498a5ec32b6f9b6e78a-a63b81d73ffd439e","baggage":"sentry-trace_id=1df619fe0f414498a5ec32b6f9b6e78a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9FSP9MJSACFYFKR43D4BXJ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:29.578Z","job_size_bytes":65,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-cc38d601c59bcd89374f015e: start","job_status":"start","queue_duration_s":0.185945,"scheduling_latency_s":0.185967}2066vm-test-run-nixbot-gitlab> gitlab # [  201.866168] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:29.784Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"cc38d601c59bcd89374f015e","created_at":"2026-06-29T10:46:29.573Z","trace_propagation_headers":{"sentry-trace":"1df619fe0f414498a5ec32b6f9b6e78a-a63b81d73ffd439e","baggage":"sentry-trace_id=1df619fe0f414498a5ec32b6f9b6e78a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9FSP9MJSACFYFKR43D4BXJ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:29.578Z","job_size_bytes":65,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-cc38d601c59bcd89374f015e: done: 0.020236 sec","job_status":"done","queue_duration_s":0.185945,"scheduling_latency_s":0.185967,"redis_calls":7,"redis_duration_s":0.0015970000000000001,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001157,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00044,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012813,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020236,"completed_at":"2026-06-29T10:46:29.784Z","load_balancing_strategy":"primary","db_duration_s":0.002755,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2067vm-test-run-nixbot-gitlab> gitlab # [  201.897120] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:29.786Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"20385e8f9c56ef0f87618eec","created_at":"2026-06-29T10:46:29.579Z","trace_propagation_headers":{"sentry-trace":"1df619fe0f414498a5ec32b6f9b6e78a-a63b81d73ffd439e","baggage":"sentry-trace_id=1df619fe0f414498a5ec32b6f9b6e78a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9FSP9MJSACFYFKR43D4BXJ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:29.581Z","job_size_bytes":65,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-20385e8f9c56ef0f87618eec: start","job_status":"start","queue_duration_s":0.204598,"scheduling_latency_s":0.20462}2068vm-test-run-nixbot-gitlab> gitlab # [  201.972562] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:29.823Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"20385e8f9c56ef0f87618eec","created_at":"2026-06-29T10:46:29.579Z","trace_propagation_headers":{"sentry-trace":"1df619fe0f414498a5ec32b6f9b6e78a-a63b81d73ffd439e","baggage":"sentry-trace_id=1df619fe0f414498a5ec32b6f9b6e78a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9FSP9MJSACFYFKR43D4BXJ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:29.581Z","job_size_bytes":65,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-20385e8f9c56ef0f87618eec: done: 0.037931 sec","job_status":"done","queue_duration_s":0.204598,"scheduling_latency_s":0.20462,"redis_calls":10,"redis_duration_s":0.002255,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001194,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.001061,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022646,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.037931,"completed_at":"2026-06-29T10:46:29.823Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009482,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2069vm-test-run-nixbot-gitlab> gitlab # [  201.993244] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:29.825Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"8fde472c43c3ea14c79d1dd6","created_at":"2026-06-29T10:46:29.592Z","trace_propagation_headers":{"sentry-trace":"cd84d747e9684b57bc46840900c8ea52-a0e26a20dce54290","baggage":"sentry-trace_id=cd84d747e9684b57bc46840900c8ea52,sentry-release=Unknown"},"correlation_id":"01KW9FSPJZFY11X1NHHSMWWPMM","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:29.597Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-8fde472c43c3ea14c79d1dd6: start","job_status":"start","queue_duration_s":0.228088,"scheduling_latency_s":0.228109}2070vm-test-run-nixbot-gitlab> gitlab # [  202.061151] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:29.852Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"8fde472c43c3ea14c79d1dd6","created_at":"2026-06-29T10:46:29.592Z","trace_propagation_headers":{"sentry-trace":"cd84d747e9684b57bc46840900c8ea52-a0e26a20dce54290","baggage":"sentry-trace_id=cd84d747e9684b57bc46840900c8ea52,sentry-release=Unknown"},"correlation_id":"01KW9FSPJZFY11X1NHHSMWWPMM","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:29.597Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-8fde472c43c3ea14c79d1dd6: done: 0.027073 sec","job_status":"done","queue_duration_s":0.228088,"scheduling_latency_s":0.228109,"redis_calls":10,"redis_duration_s":0.002214,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000405,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001294,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000515,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017816,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000345575,"duration_s":0.027073,"completed_at":"2026-06-29T10:46:29.852Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003692,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2071vm-test-run-nixbot-gitlab> gitlab # [  202.144160] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:29.853Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"34cfcb300e21b1aa22687d69","created_at":"2026-06-29T10:46:29.695Z","trace_propagation_headers":{"sentry-trace":"1df619fe0f414498a5ec32b6f9b6e78a-a63b81d73ffd439e","baggage":"sentry-trace_id=1df619fe0f414498a5ec32b6f9b6e78a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9FSP9MJSACFYFKR43D4BXJ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:29.700Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-34cfcb300e21b1aa22687d69: start","job_status":"start","queue_duration_s":0.152787,"scheduling_latency_s":0.152806}2072vm-test-run-nixbot-gitlab> gitlab # [  202.145334] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:29.963Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"34cfcb300e21b1aa22687d69","created_at":"2026-06-29T10:46:29.695Z","trace_propagation_headers":{"sentry-trace":"1df619fe0f414498a5ec32b6f9b6e78a-a63b81d73ffd439e","baggage":"sentry-trace_id=1df619fe0f414498a5ec32b6f9b6e78a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9FSP9MJSACFYFKR43D4BXJ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:29.700Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-34cfcb300e21b1aa22687d69: done: 0.109527 sec","job_status":"done","queue_duration_s":0.152787,"scheduling_latency_s":0.152806,"gitaly_calls":3,"gitaly_duration_s":0.008407,"redis_calls":39,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.006534,"redis_read_bytes":71,"redis_write_bytes":5658,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000925,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":16,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.001165,"redis_cache_read_bytes":32,"redis_cache_write_bytes":1470,"redis_queues_calls":2,"redis_queues_duration_s":0.0003,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00168,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000226,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000252,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001061,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":31,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":31,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":19,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.073592,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.088157853,"duration_s":0.109527,"completed_at":"2026-06-29T10:46:29.963Z","load_balancing_strategy":"primary","db_duration_s":0.026412,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2073vm-test-run-nixbot-gitlab> gitlab # [  202.239477] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:29.964Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"d31adbb0334dc1150c9ebaa7","created_at":"2026-06-29T10:46:29.897Z","trace_propagation_headers":{"sentry-trace":"1df619fe0f414498a5ec32b6f9b6e78a-1b0fff0e9dbc4ec5","baggage":"sentry-trace_id=1df619fe0f414498a5ec32b6f9b6e78a,sentry-release=Unknown"},"correlation_id":"01KW9FSP9MJSACFYFKR43D4BXJ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:29.898Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-d31adbb0334dc1150c9ebaa7: start","job_status":"start","queue_duration_s":0.065749,"scheduling_latency_s":0.06577}2074vm-test-run-nixbot-gitlab> gitlab # [  202.240990] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:29.987Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"d31adbb0334dc1150c9ebaa7","created_at":"2026-06-29T10:46:29.897Z","trace_propagation_headers":{"sentry-trace":"1df619fe0f414498a5ec32b6f9b6e78a-1b0fff0e9dbc4ec5","baggage":"sentry-trace_id=1df619fe0f414498a5ec32b6f9b6e78a,sentry-release=Unknown"},"correlation_id":"01KW9FSP9MJSACFYFKR43D4BXJ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:29.898Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-d31adbb0334dc1150c9ebaa7: done: 0.022647 sec","job_status":"done","queue_duration_s":0.065749,"scheduling_latency_s":0.06577,"redis_calls":7,"redis_duration_s":0.001623,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001144,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000479,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015758,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022647,"completed_at":"2026-06-29T10:46:29.987Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002849,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2075vm-test-run-nixbot-gitlab> gitlab # [  202.285646] gitaly[1389]: time="2026-06-29T10:46:29.908Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSP9MJSACFYFKR43D4BXJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-29T10:46:59.806" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:29.907" grpc.time_ms=0.916 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2076vm-test-run-nixbot-gitlab> gitlab # [  202.369329] gitaly[1389]: time="2026-06-29T10:46:29.933Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSP9MJSACFYFKR43D4BXJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T10:46:59.831" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:29.932" grpc.time_ms=0.634 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2077vm-test-run-nixbot-gitlab> gitlab # [  202.370755] gitaly[1389]: time="2026-06-29T10:46:29.955Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSP9MJSACFYFKR43D4BXJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T10:46:59.853" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:29.954" grpc.time_ms=0.595 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2078vm-test-run-nixbot-gitlab> gitlab # [  202.424705] gitaly[1389]: time="2026-06-29T10:46:30.398Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSQK5TZJYJE7CQQNAFQTT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T10:47:00.197" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:30.397" grpc.time_ms=0.666 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2079vm-test-run-nixbot-gitlab> gitlab # [  202.472316] gitaly[1389]: time="2026-06-29T10:46:30.445Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSQK5TZJYJE7CQQNAFQTT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T10:46:40.221" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T10:46:30.432" grpc.time_ms=13.199 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2080vm-test-run-nixbot-gitlab> gitlab # [  202.505870] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:30.479Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"81dcca651906a8278e57761a","created_at":"2026-06-29T10:46:30.474Z","trace_propagation_headers":{"sentry-trace":"1e9355543e7245d48bec48938d6235f0-5b210be2267544eb","baggage":"sentry-trace_id=1e9355543e7245d48bec48938d6235f0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9FSQK5TZJYJE7CQQNAFQTT","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:30.477Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-81dcca651906a8278e57761a: start","job_status":"start","queue_duration_s":0.001857,"scheduling_latency_s":0.001894}2081vm-test-run-nixbot-gitlab> gitlab # [  202.545381] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:30 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/f55b93e3be80c71b3412c6f12756acbfb61aa22b HTTP/1.1" 201 651 "" "python-httpx/0.28.1" 7842082vm-test-run-nixbot-gitlab> gitlab # [  202.550549] nixbot[3048]: {"ts": "2026-06-29T10:46:30.523Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/f55b93e3be80c71b3412c6f12756acbfb61aa22b \"HTTP/1.1 201 Created\""}2083vm-test-run-nixbot-gitlab> gitlab # [  202.619375] gitaly[1389]: time="2026-06-29T10:46:30.593Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSRC0XQW509016GPQDX84 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T10:47:00.391" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:30.592" grpc.time_ms=0.657 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2084vm-test-run-nixbot-gitlab> gitlab # [  202.651535] gitaly[1389]: time="2026-06-29T10:46:30.624Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9FSQK5TZJYJE7CQQNAFQTT grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T10:46:30.624" grpc.time_ms=0.377 pid=1389 span.kind=server system=grpc2085vm-test-run-nixbot-gitlab> gitlab # [  202.677929] gitaly[1389]: time="2026-06-29T10:46:30.651Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9FSQK5TZJYJE7CQQNAFQTT grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3899 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T10:46:30.650" grpc.time_ms=1.201 pid=1389 span.kind=server system=grpc2086vm-test-run-nixbot-gitlab> gitlab # [  202.706832] gitaly[1389]: time="2026-06-29T10:46:30.680Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSRC0XQW509016GPQDX84 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T10:46:40.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T10:46:30.674" grpc.time_ms=6.387 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2087vm-test-run-nixbot-gitlab> gitlab # [  202.739329] gitaly[1389]: time="2026-06-29T10:46:30.713Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9FSQK5TZJYJE7CQQNAFQTT grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T10:46:30.712" grpc.time_ms=0.488 pid=1389 span.kind=server system=grpc2088vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee1f362830143952a51bf976a49d3d77debbfb0c/statuses2089vm-test-run-nixbot-gitlab> gitlab # [  202.765487] gitaly[1389]: time="2026-06-29T10:46:30.739Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSQK5TZJYJE7CQQNAFQTT diskcache=1adf6bf2-859b-463d-8b9f-e11dcf8c8d7b grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-29T10:47:00.494" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-29T10:46:30.595" pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2090vm-test-run-nixbot-gitlab> gitlab # [  202.767593] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:30 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/f55b93e3be80c71b3412c6f12756acbfb61aa22b HTTP/1.1" 201 665 "" "python-httpx/0.28.1" 2122091vm-test-run-nixbot-gitlab> gitlab # [  202.769187] nixbot[3048]: {"ts": "2026-06-29T10:46:30.742Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/f55b93e3be80c71b3412c6f12756acbfb61aa22b \"HTTP/1.1 201 Created\""}2092vm-test-run-nixbot-gitlab> gitlab # [  202.790141] gitaly[1389]: time="2026-06-29T10:46:30.763Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=103 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7235 command.oublock=0 command.real_time_ms=134 command.spawn_token_fork_ms=0 command.system_time_ms=66 command.user_time_ms=37 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSQK5TZJYJE7CQQNAFQTT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-29T10:47:00.494" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T10:46:30.595" grpc.time_ms=144.304 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2093vm-test-run-nixbot-gitlab> gitlab # [  202.840969] gitaly[1389]: time="2026-06-29T10:46:30.814Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSRK70Z19XZW25733RVY8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T10:47:00.113" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:30.813" grpc.time_ms=0.74 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2094vm-test-run-nixbot-gitlab> gitlab # [  202.876724] gitaly[1389]: time="2026-06-29T10:46:30.850Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSQK5TZJYJE7CQQNAFQTT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-29T10:47:00.748" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:30.849" grpc.time_ms=0.972 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2095vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee1f362830143952a51bf976a49d3d77debbfb0c/statuses, in 0.17 seconds)2096vm-test-run-nixbot-gitlab> [{'id': 1, 'sha': 'ee1f362830143952a51bf976a49d3d77debbfb0c', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-29T10:46:29.369Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2097vm-test-run-nixbot-gitlab> gitlab # [  202.922427] gitaly[1389]: time="2026-06-29T10:46:30.896Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSQK5TZJYJE7CQQNAFQTT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T10:47:00.792" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:30.894" grpc.time_ms=1.261 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2098vm-test-run-nixbot-gitlab> gitlab # [  202.926109] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:30 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/ee1f362830143952a51bf976a49d3d77debbfb0c/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1392099vm-test-run-nixbot-gitlab> gitlab # [  203.016801] gitaly[1389]: time="2026-06-29T10:46:30.990Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSQK5TZJYJE7CQQNAFQTT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T10:47:00.887" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:30.988" grpc.time_ms=1.451 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2100vm-test-run-nixbot-gitlab> gitlab # [  203.043944] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:31.014Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"3c1bef94482860b4e2d2ea03","created_at":"2026-06-29T10:46:31.004Z","trace_propagation_headers":{"sentry-trace":"1e9355543e7245d48bec48938d6235f0-6a197ec5b8e14ba7","baggage":"sentry-trace_id=1e9355543e7245d48bec48938d6235f0,sentry-release=Unknown"},"correlation_id":"01KW9FSQK5TZJYJE7CQQNAFQTT","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"81dcca651906a8278e57761a","job_size_bytes":3,"pid":2114,"job_status":"deduplicated","message":"PipelineProcessWorker JID-3c1bef94482860b4e2d2ea03: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}2101vm-test-run-nixbot-gitlab> gitlab # [  203.046971] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:31.020Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"81dcca651906a8278e57761a","created_at":"2026-06-29T10:46:30.474Z","trace_propagation_headers":{"sentry-trace":"1e9355543e7245d48bec48938d6235f0-5b210be2267544eb","baggage":"sentry-trace_id=1e9355543e7245d48bec48938d6235f0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9FSQK5TZJYJE7CQQNAFQTT","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:30.477Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2114,"job_status":"rescheduled","message":"PipelineProcessWorker JID-81dcca651906a8278e57761a: rescheduled"}2102vm-test-run-nixbot-gitlab> gitlab # [  203.061636] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:31.034Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"81dcca651906a8278e57761a","created_at":"2026-06-29T10:46:30.474Z","trace_propagation_headers":{"sentry-trace":"1e9355543e7245d48bec48938d6235f0-5b210be2267544eb","baggage":"sentry-trace_id=1e9355543e7245d48bec48938d6235f0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9FSQK5TZJYJE7CQQNAFQTT","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:30.477Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-81dcca651906a8278e57761a: done: 0.555288 sec","job_status":"done","queue_duration_s":0.001857,"scheduling_latency_s":0.001894,"gitaly_calls":4,"gitaly_duration_s":0.211106,"redis_calls":104,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.055687,"redis_read_bytes":264,"redis_write_bytes":24006,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.002702,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":17,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.003276,"redis_cache_read_bytes":33,"redis_cache_write_bytes":1526,"redis_feature_flag_calls":22,"redis_feature_flag_duration_s":0.026658,"redis_feature_flag_read_bytes":22,"redis_feature_flag_write_bytes":3871,"redis_queues_calls":22,"redis_queues_duration_s":0.004849,"redis_queues_read_bytes":24,"redis_queues_write_bytes":12385,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.010403,"redis_queues_metadata_read_bytes":110,"redis_queues_metadata_write_bytes":3546,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000389,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000844,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003864,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":850,"db_count":47,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.071,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.025,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.03,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.041,"db_ci_duration_s":0.03,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.221295,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.497793994,"duration_s":0.555288,"completed_at":"2026-06-29T10:46:31.034Z","load_balancing_strategy":"primary","db_duration_s":0.078052,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2103vm-test-run-nixbot-gitlab> gitlab # [  203.179693] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:31.038Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"97a80146e2b682dc948140c5","created_at":"2026-06-29T10:46:30.576Z","trace_propagation_headers":{"sentry-trace":"1e9355543e7245d48bec48938d6235f0-6a197ec5b8e14ba7","baggage":"sentry-trace_id=1e9355543e7245d48bec48938d6235f0,sentry-release=Unknown"},"correlation_id":"01KW9FSQK5TZJYJE7CQQNAFQTT","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:30.583Z","job_size_bytes":23,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-97a80146e2b682dc948140c5: start","job_status":"start","queue_duration_s":0.454617,"scheduling_latency_s":0.454674}2104vm-test-run-nixbot-gitlab> gitlab # [  203.188862] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:31.162Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"97a80146e2b682dc948140c5","created_at":"2026-06-29T10:46:30.576Z","trace_propagation_headers":{"sentry-trace":"1e9355543e7245d48bec48938d6235f0-6a197ec5b8e14ba7","baggage":"sentry-trace_id=1e9355543e7245d48bec48938d6235f0,sentry-release=Unknown"},"correlation_id":"01KW9FSQK5TZJYJE7CQQNAFQTT","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:30.583Z","job_size_bytes":23,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-97a80146e2b682dc948140c5: done: 0.124266 sec","job_status":"done","queue_duration_s":0.454617,"scheduling_latency_s":0.454674,"redis_calls":15,"redis_duration_s":0.006894,"redis_read_bytes":13,"redis_write_bytes":3012,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000854,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_calls":2,"redis_queues_duration_s":0.000571,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1627,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003415,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":742,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.001085,"redis_rate_limiting_write_bytes":148,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000969,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":13,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":13,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.090543,"worker_id":"sidekiq_0","rate_limiting_gates":["permanent_email_failure","temporary_email_failure"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000517664,"duration_s":0.124266,"completed_at":"2026-06-29T10:46:31.162Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.019891,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2105vm-test-run-nixbot-gitlab> gitlab # [  203.220973] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:31.164Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"b433279bdad4749607e0ae30","created_at":"2026-06-29T10:46:30.769Z","trace_propagation_headers":{"sentry-trace":"1e9355543e7245d48bec48938d6235f0-6a197ec5b8e14ba7","baggage":"sentry-trace_id=1e9355543e7245d48bec48938d6235f0,sentry-release=Unknown"},"correlation_id":"01KW9FSQK5TZJYJE7CQQNAFQTT","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:30.774Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-b433279bdad4749607e0ae30: start","job_status":"start","queue_duration_s":0.390129,"scheduling_latency_s":0.390153}2106vm-test-run-nixbot-gitlab> gitlab # [  203.247345] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:31.195Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"b433279bdad4749607e0ae30","created_at":"2026-06-29T10:46:30.769Z","trace_propagation_headers":{"sentry-trace":"1e9355543e7245d48bec48938d6235f0-6a197ec5b8e14ba7","baggage":"sentry-trace_id=1e9355543e7245d48bec48938d6235f0,sentry-release=Unknown"},"correlation_id":"01KW9FSQK5TZJYJE7CQQNAFQTT","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:30.774Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-b433279bdad4749607e0ae30: done: 0.031163 sec","job_status":"done","queue_duration_s":0.390129,"scheduling_latency_s":0.390153,"redis_calls":10,"redis_duration_s":0.004564,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000891,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002634,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001039,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016368,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000519061,"duration_s":0.031163,"completed_at":"2026-06-29T10:46:31.195Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005808,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2107vm-test-run-nixbot-gitlab> gitlab # [  203.326176] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:31.198Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"81e6ccea190a6ccc81c7210f","created_at":"2026-06-29T10:46:30.784Z","trace_propagation_headers":{"sentry-trace":"1e9355543e7245d48bec48938d6235f0-6a197ec5b8e14ba7","baggage":"sentry-trace_id=1e9355543e7245d48bec48938d6235f0,sentry-release=Unknown"},"correlation_id":"01KW9FSQK5TZJYJE7CQQNAFQTT","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:30.790Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-81e6ccea190a6ccc81c7210f: start","job_status":"start","queue_duration_s":0.407295,"scheduling_latency_s":0.407318}2108vm-test-run-nixbot-gitlab> gitlab # [  203.394654] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:31.250Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"81e6ccea190a6ccc81c7210f","created_at":"2026-06-29T10:46:30.784Z","trace_propagation_headers":{"sentry-trace":"1e9355543e7245d48bec48938d6235f0-6a197ec5b8e14ba7","baggage":"sentry-trace_id=1e9355543e7245d48bec48938d6235f0,sentry-release=Unknown"},"correlation_id":"01KW9FSQK5TZJYJE7CQQNAFQTT","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:30.790Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-81e6ccea190a6ccc81c7210f: done: 0.052346 sec","job_status":"done","queue_duration_s":0.407295,"scheduling_latency_s":0.407318,"redis_calls":10,"redis_duration_s":0.004425,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000732,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00295,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000743,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034911,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000673549,"duration_s":0.052346,"completed_at":"2026-06-29T10:46:31.250Z","load_balancing_strategy":"primary","db_duration_s":0.007239,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2109vm-test-run-nixbot-gitlab> gitlab # [  203.411980] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:31.251Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"96b63e6f2550360ed1dfc9be","created_at":"2026-06-29T10:46:30.800Z","trace_propagation_headers":{"sentry-trace":"1e9355543e7245d48bec48938d6235f0-6a197ec5b8e14ba7","baggage":"sentry-trace_id=1e9355543e7245d48bec48938d6235f0,sentry-release=Unknown"},"correlation_id":"01KW9FSQK5TZJYJE7CQQNAFQTT","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:30.803Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-96b63e6f2550360ed1dfc9be: start","job_status":"start","queue_duration_s":0.44821,"scheduling_latency_s":0.448232}2110vm-test-run-nixbot-gitlab> gitlab # [  203.438365] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:31.277Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"96b63e6f2550360ed1dfc9be","created_at":"2026-06-29T10:46:30.800Z","trace_propagation_headers":{"sentry-trace":"1e9355543e7245d48bec48938d6235f0-6a197ec5b8e14ba7","baggage":"sentry-trace_id=1e9355543e7245d48bec48938d6235f0,sentry-release=Unknown"},"correlation_id":"01KW9FSQK5TZJYJE7CQQNAFQTT","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:30.803Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-96b63e6f2550360ed1dfc9be: done: 0.025942 sec","job_status":"done","queue_duration_s":0.44821,"scheduling_latency_s":0.448232,"redis_calls":13,"redis_duration_s":0.002929,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00043,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001396,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000654,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000449,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016519,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000336355,"duration_s":0.025942,"completed_at":"2026-06-29T10:46:31.277Z","load_balancing_strategy":"primary","db_duration_s":0.003374,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2111vm-test-run-nixbot-gitlab> gitlab # [  203.514117] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:31.279Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"99a6919da65976f12b880924","created_at":"2026-06-29T10:46:31.151Z","trace_propagation_headers":{"sentry-trace":"1e9355543e7245d48bec48938d6235f0-e9a03858e6d94748","baggage":"sentry-trace_id=1e9355543e7245d48bec48938d6235f0,sentry-release=Unknown"},"correlation_id":"01KW9FSQK5TZJYJE7CQQNAFQTT","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:06503e90d66a58e670d4bef8c2fde69433c3310a14ddb55df19ff3aaef10f8d5","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:31.154Z","job_size_bytes":464,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-99a6919da65976f12b880924: start","job_status":"start","queue_duration_s":0.12495,"scheduling_latency_s":0.124978}2112vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee1f362830143952a51bf976a49d3d77debbfb0c/statuses2113vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee1f362830143952a51bf976a49d3d77debbfb0c/statuses, in 0.10 seconds)2114vm-test-run-nixbot-gitlab> [{'id': 1, 'sha': 'ee1f362830143952a51bf976a49d3d77debbfb0c', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-29T10:46:29.369Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2115vm-test-run-nixbot-gitlab> gitlab # [  203.535712] sidekiq-cluster[2114]: {"severity":"WARN","time":"2026-06-29T10:46:31.308Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"99a6919da65976f12b880924","created_at":"2026-06-29T10:46:31.151Z","trace_propagation_headers":{"sentry-trace":"1e9355543e7245d48bec48938d6235f0-e9a03858e6d94748","baggage":"sentry-trace_id=1e9355543e7245d48bec48938d6235f0,sentry-release=Unknown"},"correlation_id":"01KW9FSQK5TZJYJE7CQQNAFQTT","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:06503e90d66a58e670d4bef8c2fde69433c3310a14ddb55df19ff3aaef10f8d5","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:31.154Z","job_size_bytes":464,"pid":2114,"sidekiq_tid":"m432","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-99a6919da65976f12b880924: fail: 0.0294 sec","job_status":"fail","queue_duration_s":0.12495,"scheduling_latency_s":0.124978,"redis_calls":7,"redis_duration_s":0.001753,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001281,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000472,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01892,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0294,"completed_at":"2026-06-29T10:46:31.308Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/pipelines.rb:33:in `pipeline_mail'","app/mailers/emails/pipelines.rb:10:in `pipeline_failed_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.004679,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2116vm-test-run-nixbot-gitlab> gitlab # [  204.191849] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:31.312Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"59dba14d83509006513ec929","created_at":"2026-06-29T10:46:30.804Z","trace_propagation_headers":{"sentry-trace":"1e9355543e7245d48bec48938d6235f0-6a197ec5b8e14ba7","baggage":"sentry-trace_id=1e9355543e7245d48bec48938d6235f0,sentry-release=Unknown"},"correlation_id":"01KW9FSQK5TZJYJE7CQQNAFQTT","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:30.810Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-59dba14d83509006513ec929: start","job_status":"start","queue_duration_s":0.501817,"scheduling_latency_s":0.501872}2117vm-test-run-nixbot-gitlab> gitlab # [  204.193592] gitaly[1389]: time="2026-06-29T10:46:31.777Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSQK5TZJYJE7CQQNAFQTT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-29T10:47:01.675" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:31.775" grpc.time_ms=1.315 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2118vm-test-run-nixbot-gitlab> gitlab # [  204.194658] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:31 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/ee1f362830143952a51bf976a49d3d77debbfb0c/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 622119vm-test-run-nixbot-gitlab> gitlab # [  204.195811] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:31.468Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"59dba14d83509006513ec929","created_at":"2026-06-29T10:46:30.804Z","trace_propagation_headers":{"sentry-trace":"1e9355543e7245d48bec48938d6235f0-6a197ec5b8e14ba7","baggage":"sentry-trace_id=1e9355543e7245d48bec48938d6235f0,sentry-release=Unknown"},"correlation_id":"01KW9FSQK5TZJYJE7CQQNAFQTT","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:30.810Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-59dba14d83509006513ec929: done: 0.156905 sec","job_status":"done","queue_duration_s":0.501817,"scheduling_latency_s":0.501872,"redis_calls":10,"redis_duration_s":0.002675,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000618,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001547,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00051,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.126,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.124,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021892,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004048,"duration_s":0.156905,"completed_at":"2026-06-29T10:46:31.468Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.125926,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2120vm-test-run-nixbot-gitlab> gitlab # [  204.266827] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:31.470Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"9ab30b0f7a4edd03357c293a","created_at":"2026-06-29T10:46:30.811Z","trace_propagation_headers":{"sentry-trace":"1e9355543e7245d48bec48938d6235f0-6a197ec5b8e14ba7","baggage":"sentry-trace_id=1e9355543e7245d48bec48938d6235f0,sentry-release=Unknown"},"correlation_id":"01KW9FSQK5TZJYJE7CQQNAFQTT","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:30.835Z","job_size_bytes":19,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-9ab30b0f7a4edd03357c293a: start","job_status":"start","queue_duration_s":0.634616,"scheduling_latency_s":0.63464}2121vm-test-run-nixbot-gitlab> gitlab # [  204.280142] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:31.611Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"9ab30b0f7a4edd03357c293a","created_at":"2026-06-29T10:46:30.811Z","trace_propagation_headers":{"sentry-trace":"1e9355543e7245d48bec48938d6235f0-6a197ec5b8e14ba7","baggage":"sentry-trace_id=1e9355543e7245d48bec48938d6235f0,sentry-release=Unknown"},"correlation_id":"01KW9FSQK5TZJYJE7CQQNAFQTT","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:30.835Z","job_size_bytes":19,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-9ab30b0f7a4edd03357c293a: done: 0.141275 sec","job_status":"done","queue_duration_s":0.634616,"scheduling_latency_s":0.63464,"redis_calls":11,"redis_duration_s":0.002809,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000541,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001798,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00047,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.112,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.11,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021575,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00039195,"duration_s":0.141275,"completed_at":"2026-06-29T10:46:31.611Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.112397,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2122vm-test-run-nixbot-gitlab> gitlab # [  204.355314] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:31.613Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"1ad252a311d932ec66562083","created_at":"2026-06-29T10:46:30.838Z","trace_propagation_headers":{"sentry-trace":"1e9355543e7245d48bec48938d6235f0-6a197ec5b8e14ba7","baggage":"sentry-trace_id=1e9355543e7245d48bec48938d6235f0,sentry-release=Unknown"},"correlation_id":"01KW9FSQK5TZJYJE7CQQNAFQTT","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5ba92b290e0434ffd4280a975048c221eb615aa5db8fd1f397623ac8e64389ed","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:30.841Z","job_size_bytes":65,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-1ad252a311d932ec66562083: start","job_status":"start","queue_duration_s":0.771518,"scheduling_latency_s":0.771542}2123vm-test-run-nixbot-gitlab> gitlab # [  204.426326] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:31.642Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"1ad252a311d932ec66562083","created_at":"2026-06-29T10:46:30.838Z","trace_propagation_headers":{"sentry-trace":"1e9355543e7245d48bec48938d6235f0-6a197ec5b8e14ba7","baggage":"sentry-trace_id=1e9355543e7245d48bec48938d6235f0,sentry-release=Unknown"},"correlation_id":"01KW9FSQK5TZJYJE7CQQNAFQTT","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5ba92b290e0434ffd4280a975048c221eb615aa5db8fd1f397623ac8e64389ed","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:30.841Z","job_size_bytes":65,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-1ad252a311d932ec66562083: done: 0.028586 sec","job_status":"done","queue_duration_s":0.771518,"scheduling_latency_s":0.771542,"redis_calls":10,"redis_duration_s":0.005317,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000463,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004323,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000531,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016036,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000375747,"duration_s":0.028586,"completed_at":"2026-06-29T10:46:31.641Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002716,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2124vm-test-run-nixbot-gitlab> gitlab # [  204.445810] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:31.643Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"8d6ca3c977b417bd378f974a","created_at":"2026-06-29T10:46:30.842Z","trace_propagation_headers":{"sentry-trace":"1e9355543e7245d48bec48938d6235f0-6a197ec5b8e14ba7","baggage":"sentry-trace_id=1e9355543e7245d48bec48938d6235f0,sentry-release=Unknown"},"correlation_id":"01KW9FSQK5TZJYJE7CQQNAFQTT","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:30.844Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-8d6ca3c977b417bd378f974a: start","job_status":"start","queue_duration_s":0.799418,"scheduling_latency_s":0.799442}2125vm-test-run-nixbot-gitlab> gitlab # [  204.471989] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:31.690Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"8d6ca3c977b417bd378f974a","created_at":"2026-06-29T10:46:30.842Z","trace_propagation_headers":{"sentry-trace":"1e9355543e7245d48bec48938d6235f0-6a197ec5b8e14ba7","baggage":"sentry-trace_id=1e9355543e7245d48bec48938d6235f0,sentry-release=Unknown"},"correlation_id":"01KW9FSQK5TZJYJE7CQQNAFQTT","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:30.844Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-8d6ca3c977b417bd378f974a: done: 0.047129 sec","job_status":"done","queue_duration_s":0.799418,"scheduling_latency_s":0.799442,"redis_calls":7,"redis_duration_s":0.002642,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002094,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000548,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036317,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.047129,"completed_at":"2026-06-29T10:46:31.690Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003828,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2126vm-test-run-nixbot-gitlab> gitlab # [  204.543124] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:31.693Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"acacffdd9bf228eb514bc79e","created_at":"2026-06-29T10:46:30.921Z","trace_propagation_headers":{"sentry-trace":"1e9355543e7245d48bec48938d6235f0-6a197ec5b8e14ba7","baggage":"sentry-trace_id=1e9355543e7245d48bec48938d6235f0,sentry-release=Unknown"},"correlation_id":"01KW9FSQK5TZJYJE7CQQNAFQTT","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:30.934Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-acacffdd9bf228eb514bc79e: start","job_status":"start","queue_duration_s":0.758734,"scheduling_latency_s":0.75878}2127vm-test-run-nixbot-gitlab> gitlab # [  204.624511] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:31.726Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"acacffdd9bf228eb514bc79e","created_at":"2026-06-29T10:46:30.921Z","trace_propagation_headers":{"sentry-trace":"1e9355543e7245d48bec48938d6235f0-6a197ec5b8e14ba7","baggage":"sentry-trace_id=1e9355543e7245d48bec48938d6235f0,sentry-release=Unknown"},"correlation_id":"01KW9FSQK5TZJYJE7CQQNAFQTT","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:30.934Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-acacffdd9bf228eb514bc79e: done: 0.033021 sec","job_status":"done","queue_duration_s":0.758734,"scheduling_latency_s":0.75878,"redis_calls":10,"redis_duration_s":0.002666,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000432,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001733,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000501,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018662,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000363454,"duration_s":0.033021,"completed_at":"2026-06-29T10:46:31.726Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006453,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2128vm-test-run-nixbot-gitlab> gitlab # [  204.632721] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:31.727Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"013a608ea9dd0fdff69919cc","created_at":"2026-06-29T10:46:30.935Z","trace_propagation_headers":{"sentry-trace":"1e9355543e7245d48bec48938d6235f0-6a197ec5b8e14ba7","baggage":"sentry-trace_id=1e9355543e7245d48bec48938d6235f0,sentry-release=Unknown"},"correlation_id":"01KW9FSQK5TZJYJE7CQQNAFQTT","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:30.943Z","job_size_bytes":6,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-013a608ea9dd0fdff69919cc: start","job_status":"start","queue_duration_s":0.783556,"scheduling_latency_s":0.783577}2129vm-test-run-nixbot-gitlab> gitlab # [  204.658104] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:31.745Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"013a608ea9dd0fdff69919cc","created_at":"2026-06-29T10:46:30.935Z","trace_propagation_headers":{"sentry-trace":"1e9355543e7245d48bec48938d6235f0-6a197ec5b8e14ba7","baggage":"sentry-trace_id=1e9355543e7245d48bec48938d6235f0,sentry-release=Unknown"},"correlation_id":"01KW9FSQK5TZJYJE7CQQNAFQTT","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:30.943Z","job_size_bytes":6,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-013a608ea9dd0fdff69919cc: done: 0.018075 sec","job_status":"done","queue_duration_s":0.783556,"scheduling_latency_s":0.783577,"redis_calls":10,"redis_duration_s":0.002192,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000536,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001251,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000405,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011023,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000340267,"duration_s":0.018075,"completed_at":"2026-06-29T10:46:31.745Z","load_balancing_strategy":"primary","db_duration_s":0.001502,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2130vm-test-run-nixbot-gitlab> gitlab # [  204.739367] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:31.746Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"f47200e32eadad46d9b999f9","created_at":"2026-06-29T10:46:31.020Z","trace_propagation_headers":{"sentry-trace":"1e9355543e7245d48bec48938d6235f0-6a197ec5b8e14ba7","baggage":"sentry-trace_id=1e9355543e7245d48bec48938d6235f0,sentry-release=Unknown"},"correlation_id":"01KW9FSQK5TZJYJE7CQQNAFQTT","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:31.024Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f47200e32eadad46d9b999f9: start","job_status":"start","queue_duration_s":0.722553,"scheduling_latency_s":0.722573}2131vm-test-run-nixbot-gitlab> gitlab # [  204.750847] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:31.811Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"f47200e32eadad46d9b999f9","created_at":"2026-06-29T10:46:31.020Z","trace_propagation_headers":{"sentry-trace":"1e9355543e7245d48bec48938d6235f0-6a197ec5b8e14ba7","baggage":"sentry-trace_id=1e9355543e7245d48bec48938d6235f0,sentry-release=Unknown"},"correlation_id":"01KW9FSQK5TZJYJE7CQQNAFQTT","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:31.024Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f47200e32eadad46d9b999f9: done: 0.064288 sec","job_status":"done","queue_duration_s":0.722553,"scheduling_latency_s":0.722573,"gitaly_calls":2,"gitaly_duration_s":0.005632,"redis_calls":19,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.00365,"redis_read_bytes":42,"redis_write_bytes":1891,"redis_cache_calls":8,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000593,"redis_cache_read_bytes":16,"redis_cache_write_bytes":735,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001663,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000208,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001186,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":20,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":20,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":9,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.044254,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.043762266,"duration_s":0.064288,"completed_at":"2026-06-29T10:46:31.811Z","load_balancing_strategy":"primary","db_duration_s":0.012551,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2132vm-test-run-nixbot-gitlab> gitlab # [  204.829219] gitaly[1389]: time="2026-06-29T10:46:31.800Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSQK5TZJYJE7CQQNAFQTT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T10:47:01.699" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:31.799" grpc.time_ms=0.552 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2133vm-test-run-nixbot-gitlab> gitlab # [  204.901300] gitaly[1389]: time="2026-06-29T10:46:31.982Z" level=info msg="finished unary call with code OK" catfile.duration_ms=3 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=3 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSSR0TE9J089RCRNX66B1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T10:47:01.277" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:31.978" grpc.time_ms=3.867 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2134vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee1f362830143952a51bf976a49d3d77debbfb0c/statuses2135vm-test-run-nixbot-gitlab> gitlab # [  205.154217] gitaly[1389]: time="2026-06-29T10:46:33.127Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSTTZ9VDABMH89GD2JMS3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T10:47:03.025" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:33.125" grpc.time_ms=1.325 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2136vm-test-run-nixbot-gitlab> gitlab # [  205.174848] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:33 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/ee1f362830143952a51bf976a49d3d77debbfb0c/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 922137vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee1f362830143952a51bf976a49d3d77debbfb0c/statuses, in 0.18 seconds)2138vm-test-run-nixbot-gitlab> [{'id': 1, 'sha': 'ee1f362830143952a51bf976a49d3d77debbfb0c', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-29T10:46:29.369Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2139vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee1f362830143952a51bf976a49d3d77debbfb0c/statuses2140vm-test-run-nixbot-gitlab> gitlab # [  206.333916] gitaly[1389]: time="2026-06-29T10:46:34.304Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSVZH1MHV874HNMSCMRAD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T10:47:04.202" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:34.302" grpc.time_ms=0.915 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2141vm-test-run-nixbot-gitlab> gitlab # [  206.400433] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:34 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/ee1f362830143952a51bf976a49d3d77debbfb0c/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1472142vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee1f362830143952a51bf976a49d3d77debbfb0c/statuses, in 0.19 seconds)2143vm-test-run-nixbot-gitlab> [{'id': 1, 'sha': 'ee1f362830143952a51bf976a49d3d77debbfb0c', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-29T10:46:29.369Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2144vm-test-run-nixbot-gitlab> gitlab # [  206.419672] nix-daemon[3413]: accepted connection from pid 3482, user nixbot2145vm-test-run-nixbot-gitlab> gitlab # [  206.465173] gitaly[1389]: time="2026-06-29T10:46:34.438Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSW4RHSVMTCRMC8QWSXT8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T10:47:04.336" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:34.437" grpc.time_ms=1.144 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2146vm-test-run-nixbot-gitlab> gitlab # [  206.545524] gitaly[1389]: time="2026-06-29T10:46:34.519Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSW4RHSVMTCRMC8QWSXT8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T10:46:44.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T10:46:34.505" grpc.time_ms=13.501 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2147vm-test-run-nixbot-gitlab> gitlab # [  206.581491] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:34.554Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"e1d097963afe033e1f0411c8","created_at":"2026-06-29T10:46:34.548Z","trace_propagation_headers":{"sentry-trace":"81626640c9bb4745aacc11eab1dfc2ad-67c07883bc7240a9","baggage":"sentry-trace_id=81626640c9bb4745aacc11eab1dfc2ad,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9FSW4RHSVMTCRMC8QWSXT8","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:34.551Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-e1d097963afe033e1f0411c8: start","job_status":"start","queue_duration_s":0.00259,"scheduling_latency_s":0.002656}2148vm-test-run-nixbot-gitlab> gitlab # [  206.600931] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:34 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/ee1f362830143952a51bf976a49d3d77debbfb0c HTTP/1.1" 201 669 "" "python-httpx/0.28.1" 1812149vm-test-run-nixbot-gitlab> gitlab # [  206.603588] nixbot[3048]: {"ts": "2026-06-29T10:46:34.575Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/ee1f362830143952a51bf976a49d3d77debbfb0c \"HTTP/1.1 201 Created\""}2150vm-test-run-nixbot-gitlab> gitlab # [  206.652585] gitaly[1389]: time="2026-06-29T10:46:34.626Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSWAJ8SM27N3GZ7YMFB60 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T10:47:04.524" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:34.625" grpc.time_ms=0.801 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2151vm-test-run-nixbot-gitlab> gitlab # [  206.687702] gitaly[1389]: time="2026-06-29T10:46:34.661Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9FSW4RHSVMTCRMC8QWSXT8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T10:46:34.660" grpc.time_ms=0.395 pid=1389 span.kind=server system=grpc2152vm-test-run-nixbot-gitlab> gitlab # [  206.691930] gitaly[1389]: time="2026-06-29T10:46:34.665Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSWAJ8SM27N3GZ7YMFB60 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T10:46:44.158" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T10:46:34.659" grpc.time_ms=6.333 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2153vm-test-run-nixbot-gitlab> gitlab # [  206.728375] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:34 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/ee1f362830143952a51bf976a49d3d77debbfb0c HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1232154vm-test-run-nixbot-gitlab> gitlab # [  206.729465] nixbot[3048]: {"ts": "2026-06-29T10:46:34.702Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/ee1f362830143952a51bf976a49d3d77debbfb0c \"HTTP/1.1 201 Created\""}2155vm-test-run-nixbot-gitlab> gitlab # [  206.757393] gitaly[1389]: time="2026-06-29T10:46:34.731Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9FSW4RHSVMTCRMC8QWSXT8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3899 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T10:46:34.730" grpc.time_ms=0.822 pid=1389 span.kind=server system=grpc2156vm-test-run-nixbot-gitlab> gitlab # [  206.794635] gitaly[1389]: time="2026-06-29T10:46:34.768Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9FSW4RHSVMTCRMC8QWSXT8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T10:46:34.767" grpc.time_ms=0.394 pid=1389 span.kind=server system=grpc2157vm-test-run-nixbot-gitlab> gitlab # [  206.819055] gitaly[1389]: time="2026-06-29T10:46:34.792Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSW4RHSVMTCRMC8QWSXT8 diskcache=6ddc78a3-00f6-4f7c-aef8-637a0036e4e6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-29T10:47:04.537" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-29T10:46:34.638" pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2158vm-test-run-nixbot-gitlab> gitlab # [  206.820271] gitaly[1389]: time="2026-06-29T10:46:34.793Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=128 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7389 command.oublock=0 command.real_time_ms=146 command.spawn_token_fork_ms=0 command.system_time_ms=86 command.user_time_ms=42 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSW4RHSVMTCRMC8QWSXT8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-29T10:47:04.537" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T10:46:34.638" grpc.time_ms=154.751 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2159vm-test-run-nixbot-gitlab> gitlab # [  206.856394] gitaly[1389]: time="2026-06-29T10:46:34.829Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSW4RHSVMTCRMC8QWSXT8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-29T10:47:04.728" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:34.829" grpc.time_ms=0.652 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2160vm-test-run-nixbot-gitlab> gitlab # [  206.878460] gitaly[1389]: time="2026-06-29T10:46:34.852Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSW4RHSVMTCRMC8QWSXT8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T10:47:04.751" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:34.851" grpc.time_ms=0.615 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2161vm-test-run-nixbot-gitlab> gitlab # [  206.913719] gitaly[1389]: time="2026-06-29T10:46:34.887Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSW4RHSVMTCRMC8QWSXT8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T10:47:04.786" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:34.886" grpc.time_ms=0.567 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2162vm-test-run-nixbot-gitlab> gitlab # [  206.934111] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:34.894Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"c97f390a39d44a7aca795d17","created_at":"2026-06-29T10:46:34.892Z","trace_propagation_headers":{"sentry-trace":"81626640c9bb4745aacc11eab1dfc2ad-73aa03021b364d37","baggage":"sentry-trace_id=81626640c9bb4745aacc11eab1dfc2ad,sentry-release=Unknown"},"correlation_id":"01KW9FSW4RHSVMTCRMC8QWSXT8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","duplicate-of":"e1d097963afe033e1f0411c8","job_size_bytes":3,"pid":2114,"job_status":"deduplicated","message":"PipelineProcessWorker JID-c97f390a39d44a7aca795d17: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}2163vm-test-run-nixbot-gitlab> gitlab # [  206.935829] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:34.896Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"e1d097963afe033e1f0411c8","created_at":"2026-06-29T10:46:34.548Z","trace_propagation_headers":{"sentry-trace":"81626640c9bb4745aacc11eab1dfc2ad-67c07883bc7240a9","baggage":"sentry-trace_id=81626640c9bb4745aacc11eab1dfc2ad,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9FSW4RHSVMTCRMC8QWSXT8","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:34.551Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2114,"job_status":"rescheduled","message":"PipelineProcessWorker JID-e1d097963afe033e1f0411c8: rescheduled"}2164vm-test-run-nixbot-gitlab> gitlab # [  206.937363] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:34.902Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"e1d097963afe033e1f0411c8","created_at":"2026-06-29T10:46:34.548Z","trace_propagation_headers":{"sentry-trace":"81626640c9bb4745aacc11eab1dfc2ad-67c07883bc7240a9","baggage":"sentry-trace_id=81626640c9bb4745aacc11eab1dfc2ad,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9FSW4RHSVMTCRMC8QWSXT8","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:34.551Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-e1d097963afe033e1f0411c8: done: 0.348582 sec","job_status":"done","queue_duration_s":0.00259,"scheduling_latency_s":0.002656,"gitaly_calls":4,"gitaly_duration_s":0.173332,"redis_calls":93,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.019415,"redis_read_bytes":262,"redis_write_bytes":23384,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001097,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":21,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.002177,"redis_cache_read_bytes":39,"redis_cache_write_bytes":1840,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000837,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":712,"redis_queues_calls":25,"redis_queues_duration_s":0.003854,"redis_queues_read_bytes":28,"redis_queues_write_bytes":14465,"redis_queues_metadata_calls":23,"redis_queues_metadata_duration_s":0.006976,"redis_queues_metadata_read_bytes":112,"redis_queues_metadata_write_bytes":3754,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000262,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000969,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.002146,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":38,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":38,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.031,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":23,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.006,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.008,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.015,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.127009,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.306883696,"duration_s":0.348582,"completed_at":"2026-06-29T10:46:34.902Z","load_balancing_strategy":"primary","db_duration_s":0.034513,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2165vm-test-run-nixbot-gitlab> gitlab # [  207.015536] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:34.904Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"61f84e3594fd8b1e37188f67","created_at":"2026-06-29T10:46:34.622Z","trace_propagation_headers":{"sentry-trace":"81626640c9bb4745aacc11eab1dfc2ad-73aa03021b364d37","baggage":"sentry-trace_id=81626640c9bb4745aacc11eab1dfc2ad,sentry-release=Unknown"},"correlation_id":"01KW9FSW4RHSVMTCRMC8QWSXT8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:34.624Z","job_size_bytes":23,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-61f84e3594fd8b1e37188f67: start","job_status":"start","queue_duration_s":0.279901,"scheduling_latency_s":0.279924}2166vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee1f362830143952a51bf976a49d3d77debbfb0c/statuses2167vm-test-run-nixbot-gitlab> gitlab # [  207.335172] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:35.307Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"61f84e3594fd8b1e37188f67","created_at":"2026-06-29T10:46:34.622Z","trace_propagation_headers":{"sentry-trace":"81626640c9bb4745aacc11eab1dfc2ad-73aa03021b364d37","baggage":"sentry-trace_id=81626640c9bb4745aacc11eab1dfc2ad,sentry-release=Unknown"},"correlation_id":"01KW9FSW4RHSVMTCRMC8QWSXT8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:34.624Z","job_size_bytes":23,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-61f84e3594fd8b1e37188f67: done: 0.402763 sec","job_status":"done","queue_duration_s":0.279901,"scheduling_latency_s":0.279924,"redis_calls":7,"redis_duration_s":0.159873,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002874,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.156999,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.187,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.185,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.052324,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.402763,"completed_at":"2026-06-29T10:46:35.307Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.187115,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2168vm-test-run-nixbot-gitlab> gitlab # [  207.340300] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:35.311Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"5eebe3105c183c6c6c09672b","created_at":"2026-06-29T10:46:34.797Z","trace_propagation_headers":{"sentry-trace":"81626640c9bb4745aacc11eab1dfc2ad-73aa03021b364d37","baggage":"sentry-trace_id=81626640c9bb4745aacc11eab1dfc2ad,sentry-release=Unknown"},"correlation_id":"01KW9FSW4RHSVMTCRMC8QWSXT8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:34.799Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-5eebe3105c183c6c6c09672b: start","job_status":"start","queue_duration_s":0.511392,"scheduling_latency_s":0.51145}2169vm-test-run-nixbot-gitlab> gitlab # [  207.460247] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:35.350Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"5eebe3105c183c6c6c09672b","created_at":"2026-06-29T10:46:34.797Z","trace_propagation_headers":{"sentry-trace":"81626640c9bb4745aacc11eab1dfc2ad-73aa03021b364d37","baggage":"sentry-trace_id=81626640c9bb4745aacc11eab1dfc2ad,sentry-release=Unknown"},"correlation_id":"01KW9FSW4RHSVMTCRMC8QWSXT8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:34.799Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-5eebe3105c183c6c6c09672b: done: 0.039979 sec","job_status":"done","queue_duration_s":0.511392,"scheduling_latency_s":0.51145,"redis_calls":7,"redis_duration_s":0.003913,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002701,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001212,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023651,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.039979,"completed_at":"2026-06-29T10:46:35.350Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00546,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2170vm-test-run-nixbot-gitlab> gitlab # [  207.463960] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:35.353Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"14f3b00a4b3409c6a7bcca8f","created_at":"2026-06-29T10:46:34.805Z","trace_propagation_headers":{"sentry-trace":"81626640c9bb4745aacc11eab1dfc2ad-73aa03021b364d37","baggage":"sentry-trace_id=81626640c9bb4745aacc11eab1dfc2ad,sentry-release=Unknown"},"correlation_id":"01KW9FSW4RHSVMTCRMC8QWSXT8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:34.806Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-14f3b00a4b3409c6a7bcca8f: start","job_status":"start","queue_duration_s":0.546498,"scheduling_latency_s":0.546521}2171vm-test-run-nixbot-gitlab> gitlab # [  207.547400] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:35.514Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"14f3b00a4b3409c6a7bcca8f","created_at":"2026-06-29T10:46:34.805Z","trace_propagation_headers":{"sentry-trace":"81626640c9bb4745aacc11eab1dfc2ad-73aa03021b364d37","baggage":"sentry-trace_id=81626640c9bb4745aacc11eab1dfc2ad,sentry-release=Unknown"},"correlation_id":"01KW9FSW4RHSVMTCRMC8QWSXT8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:34.806Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-14f3b00a4b3409c6a7bcca8f: done: 0.161158 sec","job_status":"done","queue_duration_s":0.546498,"scheduling_latency_s":0.546521,"redis_calls":7,"redis_duration_s":0.003437,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002298,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001139,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.13,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.13,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023498,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.161158,"completed_at":"2026-06-29T10:46:35.514Z","load_balancing_strategy":"primary","db_duration_s":0.129814,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2172vm-test-run-nixbot-gitlab> gitlab # [  207.555941] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:35.518Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"b305dc0db4bacf9475b39e0d","created_at":"2026-06-29T10:46:34.814Z","trace_propagation_headers":{"sentry-trace":"81626640c9bb4745aacc11eab1dfc2ad-73aa03021b364d37","baggage":"sentry-trace_id=81626640c9bb4745aacc11eab1dfc2ad,sentry-release=Unknown"},"correlation_id":"01KW9FSW4RHSVMTCRMC8QWSXT8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:34.816Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-b305dc0db4bacf9475b39e0d: start","job_status":"start","queue_duration_s":0.701871,"scheduling_latency_s":0.701926}2173vm-test-run-nixbot-gitlab> gitlab # [  207.596497] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:35.569Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"b305dc0db4bacf9475b39e0d","created_at":"2026-06-29T10:46:34.814Z","trace_propagation_headers":{"sentry-trace":"81626640c9bb4745aacc11eab1dfc2ad-73aa03021b364d37","baggage":"sentry-trace_id=81626640c9bb4745aacc11eab1dfc2ad,sentry-release=Unknown"},"correlation_id":"01KW9FSW4RHSVMTCRMC8QWSXT8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:34.816Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-b305dc0db4bacf9475b39e0d: done: 0.051205 sec","job_status":"done","queue_duration_s":0.701871,"scheduling_latency_s":0.701926,"redis_calls":10,"redis_duration_s":0.00575,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002742,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001404,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001604,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030855,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.051205,"completed_at":"2026-06-29T10:46:35.569Z","load_balancing_strategy":"primary","db_duration_s":0.007562,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2174vm-test-run-nixbot-gitlab> gitlab # [  207.683215] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:35.571Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"4b0db90d54456d118948a6ad","created_at":"2026-06-29T10:46:34.816Z","trace_propagation_headers":{"sentry-trace":"81626640c9bb4745aacc11eab1dfc2ad-73aa03021b364d37","baggage":"sentry-trace_id=81626640c9bb4745aacc11eab1dfc2ad,sentry-release=Unknown"},"correlation_id":"01KW9FSW4RHSVMTCRMC8QWSXT8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:34.817Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-4b0db90d54456d118948a6ad: start","job_status":"start","queue_duration_s":0.753178,"scheduling_latency_s":0.753202}2175vm-test-run-nixbot-gitlab> gitlab # [  207.684459] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:35.591Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"4b0db90d54456d118948a6ad","created_at":"2026-06-29T10:46:34.816Z","trace_propagation_headers":{"sentry-trace":"81626640c9bb4745aacc11eab1dfc2ad-73aa03021b364d37","baggage":"sentry-trace_id=81626640c9bb4745aacc11eab1dfc2ad,sentry-release=Unknown"},"correlation_id":"01KW9FSW4RHSVMTCRMC8QWSXT8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:34.817Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-4b0db90d54456d118948a6ad: done: 0.019878 sec","job_status":"done","queue_duration_s":0.753178,"scheduling_latency_s":0.753202,"redis_calls":7,"redis_duration_s":0.001906,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001455,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000451,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011101,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019878,"completed_at":"2026-06-29T10:46:35.590Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004144,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2176vm-test-run-nixbot-gitlab> gitlab # [  207.795075] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:35.592Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"2e7c4de4d4c58d2e48c77ce6","created_at":"2026-06-29T10:46:34.818Z","trace_propagation_headers":{"sentry-trace":"81626640c9bb4745aacc11eab1dfc2ad-73aa03021b364d37","baggage":"sentry-trace_id=81626640c9bb4745aacc11eab1dfc2ad,sentry-release=Unknown"},"correlation_id":"01KW9FSW4RHSVMTCRMC8QWSXT8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:34.820Z","job_size_bytes":19,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-2e7c4de4d4c58d2e48c77ce6: start","job_status":"start","queue_duration_s":0.771965,"scheduling_latency_s":0.771986}2177vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee1f362830143952a51bf976a49d3d77debbfb0c/statuses, in 0.47 seconds)2178vm-test-run-nixbot-gitlab> [{'id': 1, 'sha': 'ee1f362830143952a51bf976a49d3d77debbfb0c', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-06-29T10:46:29.369Z', 'started_at': None, 'finished_at': '2026-06-29T10:46:34.537Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': 'ee1f362830143952a51bf976a49d3d77debbfb0c', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-06-29T10:46:34.684Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2179vm-test-run-nixbot-gitlab> gitlab # [  207.796331] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:35.611Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"2e7c4de4d4c58d2e48c77ce6","created_at":"2026-06-29T10:46:34.818Z","trace_propagation_headers":{"sentry-trace":"81626640c9bb4745aacc11eab1dfc2ad-73aa03021b364d37","baggage":"sentry-trace_id=81626640c9bb4745aacc11eab1dfc2ad,sentry-release=Unknown"},"correlation_id":"01KW9FSW4RHSVMTCRMC8QWSXT8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:34.820Z","job_size_bytes":19,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-2e7c4de4d4c58d2e48c77ce6: done: 0.018992 sec","job_status":"done","queue_duration_s":0.771965,"scheduling_latency_s":0.771986,"redis_calls":8,"redis_duration_s":0.001695,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00129,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000405,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012079,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018992,"completed_at":"2026-06-29T10:46:35.611Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002588,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2180vm-test-run-nixbot-gitlab> gitlab # [  207.871885] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:35.612Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"3c6a725a496d3652986c78fd","created_at":"2026-06-29T10:46:34.821Z","trace_propagation_headers":{"sentry-trace":"81626640c9bb4745aacc11eab1dfc2ad-73aa03021b364d37","baggage":"sentry-trace_id=81626640c9bb4745aacc11eab1dfc2ad,sentry-release=Unknown"},"correlation_id":"01KW9FSW4RHSVMTCRMC8QWSXT8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:34.822Z","job_size_bytes":66,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-3c6a725a496d3652986c78fd: start","job_status":"start","queue_duration_s":0.790172,"scheduling_latency_s":0.790194}2181vm-test-run-nixbot-gitlab> gitlab # [  207.872879] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:35.627Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"3c6a725a496d3652986c78fd","created_at":"2026-06-29T10:46:34.821Z","trace_propagation_headers":{"sentry-trace":"81626640c9bb4745aacc11eab1dfc2ad-73aa03021b364d37","baggage":"sentry-trace_id=81626640c9bb4745aacc11eab1dfc2ad,sentry-release=Unknown"},"correlation_id":"01KW9FSW4RHSVMTCRMC8QWSXT8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:34.822Z","job_size_bytes":66,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-3c6a725a496d3652986c78fd: done: 0.014836 sec","job_status":"done","queue_duration_s":0.790172,"scheduling_latency_s":0.790194,"redis_calls":7,"redis_duration_s":0.001533,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001079,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000454,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009877,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014836,"completed_at":"2026-06-29T10:46:35.627Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001112,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2182vm-test-run-nixbot-gitlab> gitlab # [  207.952270] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:35.629Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"56bf57a9d008ad77190cc8ac","created_at":"2026-06-29T10:46:34.823Z","trace_propagation_headers":{"sentry-trace":"81626640c9bb4745aacc11eab1dfc2ad-73aa03021b364d37","baggage":"sentry-trace_id=81626640c9bb4745aacc11eab1dfc2ad,sentry-release=Unknown"},"correlation_id":"01KW9FSW4RHSVMTCRMC8QWSXT8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:34.824Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-56bf57a9d008ad77190cc8ac: start","job_status":"start","queue_duration_s":0.804658,"scheduling_latency_s":0.80468}2183vm-test-run-nixbot-gitlab> gitlab # [  207.957216] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:35.651Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"56bf57a9d008ad77190cc8ac","created_at":"2026-06-29T10:46:34.823Z","trace_propagation_headers":{"sentry-trace":"81626640c9bb4745aacc11eab1dfc2ad-73aa03021b364d37","baggage":"sentry-trace_id=81626640c9bb4745aacc11eab1dfc2ad,sentry-release=Unknown"},"correlation_id":"01KW9FSW4RHSVMTCRMC8QWSXT8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:34.824Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-56bf57a9d008ad77190cc8ac: done: 0.021971 sec","job_status":"done","queue_duration_s":0.804658,"scheduling_latency_s":0.80468,"redis_calls":7,"redis_duration_s":0.001533,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001104,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000429,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015824,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021971,"completed_at":"2026-06-29T10:46:35.650Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002423,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2184vm-test-run-nixbot-gitlab> gitlab # [  208.025487] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:35.652Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"0b7dd4460841541b2ed94a97","created_at":"2026-06-29T10:46:34.858Z","trace_propagation_headers":{"sentry-trace":"81626640c9bb4745aacc11eab1dfc2ad-73aa03021b364d37","baggage":"sentry-trace_id=81626640c9bb4745aacc11eab1dfc2ad,sentry-release=Unknown"},"correlation_id":"01KW9FSW4RHSVMTCRMC8QWSXT8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:34.859Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-0b7dd4460841541b2ed94a97: start","job_status":"start","queue_duration_s":0.792711,"scheduling_latency_s":0.792733}2185vm-test-run-nixbot-gitlab> gitlab # [  208.052461] gitaly[1389]: time="2026-06-29T10:46:35.766Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9FSW4RHSVMTCRMC8QWSXT8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-29T10:47:05.665" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:35.765" grpc.time_ms=0.852 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2186vm-test-run-nixbot-gitlab> gitlab # [  208.072555] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:35 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/ee1f362830143952a51bf976a49d3d77debbfb0c/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 582187vm-test-run-nixbot-gitlab> gitlab # [  208.076508] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:35.673Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"0b7dd4460841541b2ed94a97","created_at":"2026-06-29T10:46:34.858Z","trace_propagation_headers":{"sentry-trace":"81626640c9bb4745aacc11eab1dfc2ad-73aa03021b364d37","baggage":"sentry-trace_id=81626640c9bb4745aacc11eab1dfc2ad,sentry-release=Unknown"},"correlation_id":"01KW9FSW4RHSVMTCRMC8QWSXT8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:34.859Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-0b7dd4460841541b2ed94a97: done: 0.02105 sec","job_status":"done","queue_duration_s":0.792711,"scheduling_latency_s":0.792733,"redis_calls":7,"redis_duration_s":0.0035629999999999998,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002717,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000846,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011081,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02105,"completed_at":"2026-06-29T10:46:35.673Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003793,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2188vm-test-run-nixbot-gitlab> gitlab # [  208.164371] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:35.674Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"fec9f7b5079148abb6f27996","created_at":"2026-06-29T10:46:34.860Z","trace_propagation_headers":{"sentry-trace":"81626640c9bb4745aacc11eab1dfc2ad-73aa03021b364d37","baggage":"sentry-trace_id=81626640c9bb4745aacc11eab1dfc2ad,sentry-release=Unknown"},"correlation_id":"01KW9FSW4RHSVMTCRMC8QWSXT8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:34.861Z","job_size_bytes":6,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-fec9f7b5079148abb6f27996: start","job_status":"start","queue_duration_s":0.813457,"scheduling_latency_s":0.813478}2189vm-test-run-nixbot-gitlab> gitlab # [  208.168286] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:35.687Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"fec9f7b5079148abb6f27996","created_at":"2026-06-29T10:46:34.860Z","trace_propagation_headers":{"sentry-trace":"81626640c9bb4745aacc11eab1dfc2ad-73aa03021b364d37","baggage":"sentry-trace_id=81626640c9bb4745aacc11eab1dfc2ad,sentry-release=Unknown"},"correlation_id":"01KW9FSW4RHSVMTCRMC8QWSXT8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:34.861Z","job_size_bytes":6,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-fec9f7b5079148abb6f27996: done: 0.012357 sec","job_status":"done","queue_duration_s":0.813457,"scheduling_latency_s":0.813478,"redis_calls":7,"redis_duration_s":0.001535,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001052,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000483,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007992,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012357,"completed_at":"2026-06-29T10:46:35.687Z","load_balancing_strategy":"primary","db_duration_s":0.000337,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2190vm-test-run-nixbot-gitlab> gitlab # [  208.262308] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:35.688Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"28c1fdcbd55e9c1d85bd9ef2","created_at":"2026-06-29T10:46:34.861Z","trace_propagation_headers":{"sentry-trace":"81626640c9bb4745aacc11eab1dfc2ad-73aa03021b364d37","baggage":"sentry-trace_id=81626640c9bb4745aacc11eab1dfc2ad,sentry-release=Unknown"},"correlation_id":"01KW9FSW4RHSVMTCRMC8QWSXT8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:34.864Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-28c1fdcbd55e9c1d85bd9ef2: start","job_status":"start","queue_duration_s":0.823527,"scheduling_latency_s":0.823548}2191vm-test-run-nixbot-gitlab> gitlab # [  208.263392] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:35.712Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"28c1fdcbd55e9c1d85bd9ef2","created_at":"2026-06-29T10:46:34.861Z","trace_propagation_headers":{"sentry-trace":"81626640c9bb4745aacc11eab1dfc2ad-73aa03021b364d37","baggage":"sentry-trace_id=81626640c9bb4745aacc11eab1dfc2ad,sentry-release=Unknown"},"correlation_id":"01KW9FSW4RHSVMTCRMC8QWSXT8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:34.864Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-28c1fdcbd55e9c1d85bd9ef2: done: 0.024049 sec","job_status":"done","queue_duration_s":0.823527,"scheduling_latency_s":0.823548,"redis_calls":10,"redis_duration_s":0.002118,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000405,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001267,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000446,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01516,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000341664,"duration_s":0.024049,"completed_at":"2026-06-29T10:46:35.712Z","load_balancing_strategy":"primary","db_duration_s":0.003873,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2192vm-test-run-nixbot-gitlab> gitlab # [  208.344208] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:35.714Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"915be884db227bd91bac2155","created_at":"2026-06-29T10:46:34.896Z","trace_propagation_headers":{"sentry-trace":"81626640c9bb4745aacc11eab1dfc2ad-73aa03021b364d37","baggage":"sentry-trace_id=81626640c9bb4745aacc11eab1dfc2ad,sentry-release=Unknown"},"correlation_id":"01KW9FSW4RHSVMTCRMC8QWSXT8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:34.898Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-915be884db227bd91bac2155: start","job_status":"start","queue_duration_s":0.816168,"scheduling_latency_s":0.816188}2193vm-test-run-nixbot-gitlab> gitlab # [  208.412824] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:35.836Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"915be884db227bd91bac2155","created_at":"2026-06-29T10:46:34.896Z","trace_propagation_headers":{"sentry-trace":"81626640c9bb4745aacc11eab1dfc2ad-73aa03021b364d37","baggage":"sentry-trace_id=81626640c9bb4745aacc11eab1dfc2ad,sentry-release=Unknown"},"correlation_id":"01KW9FSW4RHSVMTCRMC8QWSXT8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:34.898Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-915be884db227bd91bac2155: done: 0.122246 sec","job_status":"done","queue_duration_s":0.816168,"scheduling_latency_s":0.816188,"gitaly_calls":3,"gitaly_duration_s":0.00937,"redis_calls":44,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.009062,"redis_read_bytes":98,"redis_write_bytes":6014,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000929,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.003094,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":2,"redis_queues_duration_s":0.000277,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001654,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000282,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000948,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000949,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":20,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.083366,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.101299188,"duration_s":0.122246,"completed_at":"2026-06-29T10:46:35.836Z","load_balancing_strategy":"primary","db_duration_s":0.026353,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2194vm-test-run-nixbot-gitlab> gitlab # [  208.429887] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:35.837Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"c7bcb09b6a35856b3b23b10a","created_at":"2026-06-29T10:46:35.756Z","trace_propagation_headers":{"sentry-trace":"81626640c9bb4745aacc11eab1dfc2ad-ff5f35e89f50492e","baggage":"sentry-trace_id=81626640c9bb4745aacc11eab1dfc2ad,sentry-release=Unknown"},"correlation_id":"01KW9FSW4RHSVMTCRMC8QWSXT8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:35.757Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-c7bcb09b6a35856b3b23b10a: start","job_status":"start","queue_duration_s":0.080062,"scheduling_latency_s":0.080087}2195vm-test-run-nixbot-gitlab> gitlab # [  208.455782] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:35.863Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"c7bcb09b6a35856b3b23b10a","created_at":"2026-06-29T10:46:35.756Z","trace_propagation_headers":{"sentry-trace":"81626640c9bb4745aacc11eab1dfc2ad-ff5f35e89f50492e","baggage":"sentry-trace_id=81626640c9bb4745aacc11eab1dfc2ad,sentry-release=Unknown"},"correlation_id":"01KW9FSW4RHSVMTCRMC8QWSXT8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:35.757Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-c7bcb09b6a35856b3b23b10a: done: 0.025802 sec","job_status":"done","queue_duration_s":0.080062,"scheduling_latency_s":0.080087,"redis_calls":7,"redis_duration_s":0.0016510000000000001,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001167,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000484,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017699,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025802,"completed_at":"2026-06-29T10:46:35.863Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002693,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2196vm-test-run-nixbot-gitlab> gitlab # [  208.538058] gitaly[1389]: time="2026-06-29T10:46:35.799Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSW4RHSVMTCRMC8QWSXT8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T10:47:05.697" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:35.798" grpc.time_ms=0.698 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2197vm-test-run-nixbot-gitlab> gitlab # [  208.607277] gitaly[1389]: time="2026-06-29T10:46:35.821Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSXG50Q5ADGT2B8SS2H93 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T10:47:05.120" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:35.820" grpc.time_ms=0.535 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2198vm-test-run-nixbot-gitlab> gitlab # [  208.612986] gitaly[1389]: time="2026-06-29T10:46:35.823Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSW4RHSVMTCRMC8QWSXT8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T10:47:05.722" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:35.822" grpc.time_ms=0.446 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2199vm-test-run-nixbot-gitlab> gitlab # [  208.773953] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:36.746Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"733342ff545aafa6a4203867","created_at":"2026-06-29T10:46:20.212Z","trace_propagation_headers":{"sentry-trace":"b4031b97d6fc411a9c92532298b9a4b2-70a8b31c63be4abc","baggage":"sentry-trace_id=b4031b97d6fc411a9c92532298b9a4b2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9FSE0RVKMH7Z4NF5SP0Z7W","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:fe41a6587ca506fd951b41c6e2be51872654aa5346bbe1f9601e208ec67e3fd1","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:36.742Z","retry_count":0,"job_size_bytes":451,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-733342ff545aafa6a4203867: start","job_status":"start","queue_duration_s":0.003615,"scheduling_latency_s":0.00368}2200vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee1f362830143952a51bf976a49d3d77debbfb0c/statuses2201vm-test-run-nixbot-gitlab> gitlab # [  208.899674] sidekiq-cluster[2114]: {"severity":"WARN","time":"2026-06-29T10:46:36.872Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"733342ff545aafa6a4203867","created_at":"2026-06-29T10:46:20.212Z","trace_propagation_headers":{"sentry-trace":"b4031b97d6fc411a9c92532298b9a4b2-70a8b31c63be4abc","baggage":"sentry-trace_id=b4031b97d6fc411a9c92532298b9a4b2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9FSE0RVKMH7Z4NF5SP0Z7W","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:fe41a6587ca506fd951b41c6e2be51872654aa5346bbe1f9601e208ec67e3fd1","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:36.742Z","retry_count":0,"job_size_bytes":451,"pid":2114,"sidekiq_tid":"ma8y","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-733342ff545aafa6a4203867: fail: 0.125665 sec","job_status":"fail","queue_duration_s":0.003615,"scheduling_latency_s":0.00368,"redis_calls":7,"redis_duration_s":0.004697,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003235,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001462,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039363,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.125665,"completed_at":"2026-06-29T10:46:36.871Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.00799,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2202vm-test-run-nixbot-gitlab> gitlab # [  209.544634] gitaly[1389]: time="2026-06-29T10:46:37.517Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FSYK0SE3C2TQ5PRKDK8MP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T10:47:07.316" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:37.517" grpc.time_ms=0.701 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2203vm-test-run-nixbot-gitlab> gitlab # [  209.611431] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:37 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/ee1f362830143952a51bf976a49d3d77debbfb0c/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 6872204vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee1f362830143952a51bf976a49d3d77debbfb0c/statuses, in 0.75 seconds)2205vm-test-run-nixbot-gitlab> [{'id': 1, 'sha': 'ee1f362830143952a51bf976a49d3d77debbfb0c', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-06-29T10:46:29.369Z', 'started_at': None, 'finished_at': '2026-06-29T10:46:34.537Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': 'ee1f362830143952a51bf976a49d3d77debbfb0c', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-06-29T10:46:34.684Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2206vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee1f362830143952a51bf976a49d3d77debbfb0c/statuses2207vm-test-run-nixbot-gitlab> gitlab # [  210.732608] gitaly[1389]: time="2026-06-29T10:46:38.705Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FT09BNDP788QS80QCMTN9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T10:47:08.503" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:38.703" grpc.time_ms=1.1 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2208vm-test-run-nixbot-gitlab> gitlab # [  210.772798] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:38 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/ee1f362830143952a51bf976a49d3d77debbfb0c/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 1102209vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee1f362830143952a51bf976a49d3d77debbfb0c/statuses, in 0.16 seconds)2210vm-test-run-nixbot-gitlab> [{'id': 1, 'sha': 'ee1f362830143952a51bf976a49d3d77debbfb0c', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-06-29T10:46:29.369Z', 'started_at': None, 'finished_at': '2026-06-29T10:46:34.537Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': 'ee1f362830143952a51bf976a49d3d77debbfb0c', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-06-29T10:46:34.684Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2211vm-test-run-nixbot-gitlab> gitlab # [  211.386691] nix-daemon[3413]: accepted connection from pid 3554, user nixbot2212vm-test-run-nixbot-gitlab> gitlab # [  211.491265] gitaly[1389]: time="2026-06-29T10:46:39.464Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FT1164DEQYKF0HJGYW658 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T10:47:09.262" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:39.463" grpc.time_ms=1.256 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2213vm-test-run-nixbot-gitlab> gitlab # [  211.555730] gitaly[1389]: time="2026-06-29T10:46:39.529Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KW9FT1164DEQYKF0HJGYW658 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-29T10:46:49.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T10:46:39.519" grpc.time_ms=9.483 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2214vm-test-run-nixbot-gitlab> gitlab # [  211.612703] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:39.585Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"3d383d8b55d1f5b52a94afe3","created_at":"2026-06-29T10:46:39.577Z","trace_propagation_headers":{"sentry-trace":"7407d4b658c749eba127f62633b715ee-e61319f3525748df","baggage":"sentry-trace_id=7407d4b658c749eba127f62633b715ee,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KW9FT1164DEQYKF0HJGYW658","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-29T10:46:39.582Z","job_size_bytes":3,"pid":2114,"sidekiq_tid":"m7lu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-3d383d8b55d1f5b52a94afe3: start","job_status":"start","queue_duration_s":0.002447,"scheduling_latency_s":0.002516}2215vm-test-run-nixbot-gitlab> gitlab # [  211.647469] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:39 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/ee1f362830143952a51bf976a49d3d77debbfb0c HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 2082216vm-test-run-nixbot-gitlab> gitlab # [  211.650155] nixbot[3048]: {"ts": "2026-06-29T10:46:39.608Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/ee1f362830143952a51bf976a49d3d77debbfb0c \"HTTP/1.1 201 Created\""}2217vm-test-run-nixbot-gitlab> gitlab # [  211.749229] gitaly[1389]: time="2026-06-29T10:46:39.721Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9FT1164DEQYKF0HJGYW658 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T10:46:39.721" grpc.time_ms=0.555 pid=1389 span.kind=server system=grpc2218vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee1f362830143952a51bf976a49d3d77debbfb0c/statuses2219vm-test-run-nixbot-gitlab> gitlab # [  211.794938] gitaly[1389]: time="2026-06-29T10:46:39.768Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9FT1164DEQYKF0HJGYW658 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3899 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T10:46:39.768" grpc.time_ms=0.342 pid=1389 span.kind=server system=grpc2220vm-test-run-nixbot-gitlab> gitlab # [  211.818673] nix-daemon[3413]: accepted connection from pid 3580, user nixbot2221vm-test-run-nixbot-gitlab> gitlab # [  211.837512] gitaly[1389]: time="2026-06-29T10:46:39.811Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KW9FT1164DEQYKF0HJGYW658 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-29T10:46:39.810" grpc.time_ms=0.325 pid=1389 span.kind=server system=grpc2222vm-test-run-nixbot-gitlab> gitlab # [  211.856615] gitaly[1389]: time="2026-06-29T10:46:39.830Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01KW9FT1164DEQYKF0HJGYW658 forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-29T10:47:09.583" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-29T10:46:39.684" operations="[repack_refs]" pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2223vm-test-run-nixbot-gitlab> gitlab # [  211.860224] gitaly[1389]: time="2026-06-29T10:46:39.833Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01KW9FT1164DEQYKF0HJGYW658 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-29T10:47:09.583" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-29T10:46:39.684" pid=1389 remote_ip="2001:db8:1::1" repository_info="{false {6 703 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-06-29 10:46:21.667117587 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root2224vm-test-run-nixbot-gitlab> gitlab # [  211.864081] gitaly[1389]: time="2026-06-29T10:46:39.837Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FT1DESATHQ5R5B3CRZQC2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T10:47:09.636" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:39.836" grpc.time_ms=0.676 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2225vm-test-run-nixbot-gitlab> gitlab # [  211.866952] gitaly[1389]: time="2026-06-29T10:46:39.840Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KW9FT1164DEQYKF0HJGYW658 diskcache=c2246398-c9ae-4368-80c2-4d52089d73fb grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-29T10:47:09.583" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-29T10:46:39.684" pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2226vm-test-run-nixbot-gitlab> gitlab # [  211.870895] gitaly[1389]: time="2026-06-29T10:46:39.840Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=141 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7498 command.oublock=0 command.real_time_ms=146 command.spawn_token_fork_ms=1 command.system_time_ms=102 command.user_time_ms=39 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FT1164DEQYKF0HJGYW658 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-29T10:47:09.583" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-29T10:46:39.684" grpc.time_ms=156.429 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2227vm-test-run-nixbot-gitlab> gitlab # [  211.941543] gitaly[1389]: time="2026-06-29T10:46:39.851Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01KW9FT1164DEQYKF0HJGYW658 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-29T10:47:09.583" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-29T10:46:39.684" pid=1389 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root2228vm-test-run-nixbot-gitlab> gitlab # [  211.943084] gitaly[1389]: time="2026-06-29T10:46:39.851Z" level=info msg="optimized repository" clean_stale_data_time=16.533386ms clean_work_trees_time="93.867µs" commit_graph_time="7.822µs" correlation_id=01KW9FT1164DEQYKF0HJGYW658 gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="8.381µs" pid=1389 prune_time="7.822µs" relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="8.381µs" storage_name=default system=housekeeping total_time=16.862199ms2229vm-test-run-nixbot-gitlab> gitlab # [  211.944322] gitlab-workhorse[1291]: gitlab 2001:db8:1::1 - - [2026/06/29:10:46:39 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/ee1f362830143952a51bf976a49d3d77debbfb0c/statuses HTTP/1.1" 200 1328 "" "curl/8.20.0" 762230vm-test-run-nixbot-gitlab> gitlab # [  211.945120] gitaly[1389]: time="2026-06-29T10:46:39.851Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01KW9FT1164DEQYKF0HJGYW658 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-29T10:47:09.583" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-29T10:46:39.684" pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2231vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee1f362830143952a51bf976a49d3d77debbfb0c/statuses, in 0.22 seconds)2232vm-test-run-nixbot-gitlab> [{'id': 1, 'sha': 'ee1f362830143952a51bf976a49d3d77debbfb0c', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-06-29T10:46:29.369Z', 'started_at': None, 'finished_at': '2026-06-29T10:46:34.537Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': 'ee1f362830143952a51bf976a49d3d77debbfb0c', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-06-29T10:46:34.684Z', 'started_at': None, 'finished_at': '2026-06-29T10:46:39.565Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2233vm-test-run-nixbot-gitlab> (finished: subtest: eval and build post commit statuses, in 10.59 seconds)2234vm-test-run-nixbot-gitlab> (finished: run the VM test script, in 246.98 seconds)2235vm-test-run-nixbot-gitlab> gitlab # [  211.946274] gitaly[1389]: time="2026-06-29T10:46:39.884Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KW9FT1164DEQYKF0HJGYW658 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-29T10:47:09.783" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:39.883" grpc.time_ms=0.735 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2236vm-test-run-nixbot-gitlab> gitlab # [  211.947174] gitaly[1389]: time="2026-06-29T10:46:39.905Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FT1164DEQYKF0HJGYW658 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T10:47:09.804" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:39.904" grpc.time_ms=0.992 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2237vm-test-run-nixbot-gitlab> gitlab # [  211.967217] gitaly[1389]: time="2026-06-29T10:46:39.933Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KW9FT1164DEQYKF0HJGYW658 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-29T10:47:09.832" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-29T10:46:39.932" grpc.time_ms=0.746 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2238vm-test-run-nixbot-gitlab> test script finished in 247.04s2239vm-test-run-nixbot-gitlab> cleanup2240vm-test-run-nixbot-gitlab> kill QemuMachine (pid 13)2241vm-test-run-nixbot-gitlab> gitlab # [  212.060557] sidekiq-cluster[2114]: {"severity":"INFO","time":"2026-06-29T10:46:39.945Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"3d383d8b55d1f5b52a94afe3","created_at":"2026-06-29T10:46:39.577Z","trace_propagation_headers":{"sentry-trace":"7407d4b658c749eba127f62633b715ee-e61319f3525748df","baggage":"sentry-trace_id=7407d4b658c749eba127f62633b715ee,sentry-environment=,sentry-release=Uqemu-system-x86_64: terminating on signal 15 from pid 6 (/nix/store/60m4rxhg2fldqaak400c0lry96ijrzqn-python3-3.13.13/bin/python3.13)2242vm-test-run-nixbot-gitlab> gitlab # n2243vm-test-run-nixbot-gitlab> (finished: cleanup, in 1.21 seconds)22442245post-build step Upload coverage to codecov: ok2246Skipping codecov: project=Mic92/nixbot attr=x86_64-linux.nixbot-gitlab