nixbot

1these 13 derivations will be built:2  /nix/store/9j6d250vwcnd94rs38limm94k2hml5z9-python3.13-nixbot.drv3  /nix/store/y9r3w3icd7q2nycrvlvp70376zc0mk5r-unit-nixbot.service.drv4  /nix/store/rqlhijjki8rchj4gfin6ap339k8rciga-system-units.drv5  /nix/store/y27hhjcxw4ivqklz727c6kwxwv8d1kgc-etc.drv6  /nix/store/vmmsvq8w4j3mzz6gl9y23rqhxgnf17d7-activate.drv7  /nix/store/zf5qlxnxabmnxi03wyl2yx1xfdw6i1s0-nixos-system-gitlab-test.drv8  /nix/store/g4b18h72clyqpma9l54a19n34zj8xxkc-closure-info.drv9  /nix/store/y8669vwv586zpx4x4rxndlkzwx77pcxl-closure-info.drv10  /nix/store/67drv4pqsmhrgy0rrvlzh3w2lpw09sy2-run-nixos-vm.drv11  /nix/store/7xnf2shfkgs4rqf4w7fbb3j82qllznmn-nixos-vm.drv12  /nix/store/r6g750glvaz52x3lrbdbxq9np5n3xycf-driverConfiguration.json.drv13  /nix/store/f9jlvnys9ww9b0nshphni0alf890hsq9-nixos-test-driver-nixbot-gitlab.drv14  /nix/store/d5b8nbiigfmhybw5jys013qzfb1kx014-vm-test-run-nixbot-gitlab.drv15building '/nix/store/9j6d250vwcnd94rs38limm94k2hml5z9-python3.13-nixbot.drv' on 'ssh-ng://nix@jamie'16building '/nix/store/9j6d250vwcnd94rs38limm94k2hml5z9-python3.13-nixbot.drv'17python3.13-nixbot> Sourcing python-remove-tests-dir-hook18python3.13-nixbot> Sourcing python-catch-conflicts-hook.sh19python3.13-nixbot> Sourcing python-remove-bin-bytecode-hook.sh20python3.13-nixbot> Sourcing pypa-build-hook21python3.13-nixbot> Using pypaBuildPhase22python3.13-nixbot> Sourcing python-runtime-deps-check-hook23python3.13-nixbot> Using pythonRuntimeDepsCheckHook24python3.13-nixbot> Sourcing pypa-install-hook25python3.13-nixbot> Using pypaInstallPhase26python3.13-nixbot> Sourcing python-imports-check-hook.sh27python3.13-nixbot> Using pythonImportsCheckPhase28python3.13-nixbot> Sourcing python-namespaces-hook29python3.13-nixbot> Sourcing python-catch-conflicts-hook.sh30python3.13-nixbot> Running phase: unpackPhase31python3.13-nixbot> unpacking source archive /nix/store/w6ylc4wcrhjqry5qi8j9bb1pvdnkysc6-nixbot32python3.13-nixbot> source root is nixbot33python3.13-nixbot> setting SOURCE_DATE_EPOCH to timestamp 315619200 of file "nixbot/pyproject.toml"34python3.13-nixbot> Running phase: patchPhase35python3.13-nixbot> Running phase: updateAutotoolsGnuConfigScriptsPhase36python3.13-nixbot> Running phase: configurePhase37python3.13-nixbot> no configure script, doing nothing38python3.13-nixbot> Running phase: buildPhase39python3.13-nixbot> Executing pypaBuildPhase40python3.13-nixbot> Creating a wheel...41python3.13-nixbot> pypa build flags: --no-isolation --outdir dist/ --wheel42python3.13-nixbot> * Getting build dependencies for wheel...43python3.13-nixbot> * Building wheel...44python3.13-nixbot> Successfully built nixbot-0.0.1-py3-none-any.whl45python3.13-nixbot> Finished creating a wheel...46python3.13-nixbot> Finished executing pypaBuildPhase47python3.13-nixbot> Running phase: pythonRuntimeDepsCheckHook48python3.13-nixbot> Executing pythonRuntimeDepsCheck49python3.13-nixbot> Checking runtime dependencies for nixbot-0.0.1-py3-none-any.whl50python3.13-nixbot> Finished executing pythonRuntimeDepsCheck51python3.13-nixbot> Running phase: installPhase52python3.13-nixbot> Executing pypaInstallPhase53python3.13-nixbot> Successfully installed nixbot-0.0.1-py3-none-any.whl54python3.13-nixbot> Finished executing pypaInstallPhase55python3.13-nixbot> Running phase: pythonOutputDistPhase56python3.13-nixbot> Executing pythonOutputDistPhase57python3.13-nixbot> Finished executing pythonOutputDistPhase58python3.13-nixbot> Running phase: fixupPhase59python3.13-nixbot> shrinking RPATHs of ELF executables and libraries in /nix/store/9lahwvyrf8sbkr7g334mkcfk679r9c3a-python3.13-nixbot60python3.13-nixbot> checking for references to /build/ in /nix/store/9lahwvyrf8sbkr7g334mkcfk679r9c3a-python3.13-nixbot...61python3.13-nixbot> patching script interpreter paths in /nix/store/9lahwvyrf8sbkr7g334mkcfk679r9c3a-python3.13-nixbot62python3.13-nixbot> stripping (with command strip and flags -S -p) in  /nix/store/9lahwvyrf8sbkr7g334mkcfk679r9c3a-python3.13-nixbot/lib /nix/store/9lahwvyrf8sbkr7g334mkcfk679r9c3a-python3.13-nixbot/bin63python3.13-nixbot> shrinking RPATHs of ELF executables and libraries in /nix/store/cwkf2pymdzjahyhzii3vj16y0rmym1dc-python3.13-nixbot-dist64python3.13-nixbot> checking for references to /build/ in /nix/store/cwkf2pymdzjahyhzii3vj16y0rmym1dc-python3.13-nixbot-dist...65python3.13-nixbot> patching script interpreter paths in /nix/store/cwkf2pymdzjahyhzii3vj16y0rmym1dc-python3.13-nixbot-dist66python3.13-nixbot> Rewriting #!/nix/store/60m4rxhg2fldqaak400c0lry96ijrzqn-python3-3.13.13/bin/python3.13 to #!/nix/store/60m4rxhg2fldqaak400c0lry96ijrzqn-python3-3.13.1367python3.13-nixbot> wrapping `/nix/store/9lahwvyrf8sbkr7g334mkcfk679r9c3a-python3.13-nixbot/bin/nixbot'...68python3.13-nixbot> Executing pythonRemoveTestsDir69python3.13-nixbot> Finished executing pythonRemoveTestsDir70python3.13-nixbot> Running phase: pythonCatchConflictsPhase71python3.13-nixbot> Running phase: pythonRemoveBinBytecodePhase72python3.13-nixbot> Running phase: pythonImportsCheckPhase73python3.13-nixbot> Executing pythonImportsCheckPhase74building '/nix/store/y9r3w3icd7q2nycrvlvp70376zc0mk5r-unit-nixbot.service.drv'75unit-nixbot.service> tribuchet: building on jamie76building '/nix/store/rqlhijjki8rchj4gfin6ap339k8rciga-system-units.drv'77system-units> tribuchet: building on jamie78building '/nix/store/y27hhjcxw4ivqklz727c6kwxwv8d1kgc-etc.drv'79etc> tribuchet: building on jamie80building '/nix/store/vmmsvq8w4j3mzz6gl9y23rqhxgnf17d7-activate.drv'81activate> tribuchet: building on jamie82building '/nix/store/zf5qlxnxabmnxi03wyl2yx1xfdw6i1s0-nixos-system-gitlab-test.drv'83nixos-system-gitlab-test> tribuchet: building on jamie84nixos-system-gitlab-test> structuredAttrs is enabled85building '/nix/store/g4b18h72clyqpma9l54a19n34zj8xxkc-closure-info.drv'86closure-info> tribuchet: building on jamie87closure-info> structuredAttrs is enabled88building '/nix/store/y8669vwv586zpx4x4rxndlkzwx77pcxl-closure-info.drv'89closure-info> tribuchet: building on jamie90closure-info> structuredAttrs is enabled91building '/nix/store/67drv4pqsmhrgy0rrvlzh3w2lpw09sy2-run-nixos-vm.drv'92run-nixos-vm> tribuchet: building on jamie93building '/nix/store/7xnf2shfkgs4rqf4w7fbb3j82qllznmn-nixos-vm.drv'94nixos-vm> tribuchet: building on jamie95building '/nix/store/r6g750glvaz52x3lrbdbxq9np5n3xycf-driverConfiguration.json.drv'96driverConfiguration.json> tribuchet: building on jamie97driverConfiguration.json> structuredAttrs is enabled98building '/nix/store/f9jlvnys9ww9b0nshphni0alf890hsq9-nixos-test-driver-nixbot-gitlab.drv'99nixos-test-driver-nixbot-gitlab> tribuchet: building on jamie100nixos-test-driver-nixbot-gitlab> Running type check (enable/disable: config.skipTypeCheck)101nixos-test-driver-nixbot-gitlab> See https://nixos.org/manual/nixos/stable/#test-opt-skipTypeCheck102nixos-test-driver-nixbot-gitlab> All checks passed!103nixos-test-driver-nixbot-gitlab> Linting test script (enable/disable: config.skipLint)104nixos-test-driver-nixbot-gitlab> See https://nixos.org/manual/nixos/stable/#test-opt-skipLint105nixos-test-driver-nixbot-gitlab> All checks passed!106building '/nix/store/d5b8nbiigfmhybw5jys013qzfb1kx014-vm-test-run-nixbot-gitlab.drv' on 'ssh-ng://nix@jamie'107building '/nix/store/d5b8nbiigfmhybw5jys013qzfb1kx014-vm-test-run-nixbot-gitlab.drv'108vm-test-run-nixbot-gitlab> Machine state will be reset. To keep it, pass --keep-machine-state109vm-test-run-nixbot-gitlab> start all VLans110vm-test-run-nixbot-gitlab> (finished: start all VLans, in 0.00 seconds)111vm-test-run-nixbot-gitlab> Test will time out and terminate in 3600 seconds112vm-test-run-nixbot-gitlab> run the VM test script113vm-test-run-nixbot-gitlab> additionally exposed symbols:114vm-test-run-nixbot-gitlab>     gitlab,115vm-test-run-nixbot-gitlab>     vlan1,116vm-test-run-nixbot-gitlab>     start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh117vm-test-run-nixbot-gitlab> start all VMs118vm-test-run-nixbot-gitlab> gitlab: starting vm119vm-test-run-nixbot-gitlab> gitlab: QEMU running (pid 13)120vm-test-run-nixbot-gitlab> gitlab # Disk image does not exist, creating the virtualisation disk image...121vm-test-run-nixbot-gitlab> gitlab # Formatting '/build/vm-state-gitlab/tmp.UDlKjgP91D', fmt=raw size=1073741824122vm-test-run-nixbot-gitlab> gitlab # mke2fs 1.47.3 (8-Jul-2025)123vm-test-run-nixbot-gitlab> gitlab # Discarding device blocks:      0/262144             done124vm-test-run-nixbot-gitlab> gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes125vm-test-run-nixbot-gitlab> gitlab # Filesystem UUID: 1e24690e-db02-49d0-bfc9-d6ae9bcd003d126vm-test-run-nixbot-gitlab> gitlab # Superblock backups stored on blocks:127vm-test-run-nixbot-gitlab> gitlab # 	32768, 98304, 163840, 229376128vm-test-run-nixbot-gitlab> gitlab # 129vm-test-run-nixbot-gitlab> (finished: start all VMs, in 34.57 seconds)130vm-test-run-nixbot-gitlab> gitlab # Allocating group tables: 0/8   done131vm-test-run-nixbot-gitlab> gitlab: waiting for unit setup-gitlab.service132vm-test-run-nixbot-gitlab> gitlab # Writing inode tables: 0/8   done133vm-test-run-nixbot-gitlab> gitlab: waiting for the VM to finish booting134vm-test-run-nixbot-gitlab> gitlab # Creating journal (8192 blocks): done135vm-test-run-nixbot-gitlab> gitlab # Writing superblocks and filesystem accounting information: 0/8   done136vm-test-run-nixbot-gitlab> gitlab # 137vm-test-run-nixbot-gitlab> gitlab # Virtualisation disk image created.138vm-test-run-nixbot-gitlab> gitlab # Creating Nix store image...139vm-test-run-nixbot-gitlab> gitlab # Created Nix store image.140vm-test-run-nixbot-gitlab> gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)141vm-test-run-nixbot-gitlab> gitlab # 142vm-test-run-nixbot-gitlab> gitlab # 143vm-test-run-nixbot-gitlab> gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA00144vm-test-run-nixbot-gitlab> gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)...145vm-test-run-nixbot-gitlab> gitlab # 146vm-test-run-nixbot-gitlab> gitlab # 147vm-test-run-nixbot-gitlab> gitlab # 148vm-test-run-nixbot-gitlab> gitlab # 149vm-test-run-nixbot-gitlab> gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB00150vm-test-run-nixbot-gitlab> gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)...151vm-test-run-nixbot-gitlab> gitlab # 152vm-test-run-nixbot-gitlab> gitlab # 153vm-test-run-nixbot-gitlab> gitlab # Booting from ROM...154vm-test-run-nixbot-gitlab> gitlab # Probing EDD (edd=off to disable)... ok155vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Linux version 6.18.34 (nixbld@localhost) (gcc (GCC) 15.2.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Jun  1 15:51:08 UTC 2026156vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/igb97j0wm8q7h3ykz6g8nlk205nm0n7q-nixos-system-gitlab-test/init regInfo=/nix/store/zhf5qch3k3702akbp2aks0v6is94rga1-closure-info/registration console=ttyS0,115200n8 console=tty0157vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-provided physical RAM map:158vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable159vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved160vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved161vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable162vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved163vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved164vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved165vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable166vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved167vm-test-run-nixbot-gitlab> gitlab # [    0.000000] NX (Execute Disable) protection: active168vm-test-run-nixbot-gitlab> gitlab # [    0.000000] APIC: Static calls initialized169vm-test-run-nixbot-gitlab> gitlab # [    0.000000] SMBIOS 2.8 present.170vm-test-run-nixbot-gitlab> gitlab # [    0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014171vm-test-run-nixbot-gitlab> gitlab # [    0.000000] DMI: Memory slots populated: 1/1172vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Hypervisor detected: KVM173vm-test-run-nixbot-gitlab> gitlab # [    0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000174vm-test-run-nixbot-gitlab> gitlab # [    0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00175vm-test-run-nixbot-gitlab> gitlab # [    0.000000] kvm-clock: using sched offset of 519833511 cycles176vm-test-run-nixbot-gitlab> gitlab # [    0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns177vm-test-run-nixbot-gitlab> gitlab # [    0.000005] tsc: Detected 2400.010 MHz processor178vm-test-run-nixbot-gitlab> gitlab # [    0.000811] last_pfn = 0x1c0000 max_arch_pfn = 0x10000000000179vm-test-run-nixbot-gitlab> gitlab # [    0.000847] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs180vm-test-run-nixbot-gitlab> gitlab # [    0.000850] x86/PAT: Configuration [0-7]: WB  WC  UC- UC  WB  WP  UC- WT181vm-test-run-nixbot-gitlab> gitlab # [    0.000902] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000182vm-test-run-nixbot-gitlab> gitlab # [    0.002753] found SMP MP-table at [mem 0x000f5470-0x000f547f]183vm-test-run-nixbot-gitlab> gitlab # [    0.002765] Using GB pages for direct mapping184vm-test-run-nixbot-gitlab> gitlab # [    0.002868] RAMDISK: [mem 0xbe4e0000-0xbffcffff]185vm-test-run-nixbot-gitlab> gitlab # [    0.002876] ACPI: Early table checksum verification disabled186vm-test-run-nixbot-gitlab> gitlab # [    0.002878] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )187vm-test-run-nixbot-gitlab> gitlab # [    0.002882] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS  BXPC     00000001 BXPC 00000001)188vm-test-run-nixbot-gitlab> gitlab # [    0.002886] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS  BXPC     00000001 BXPC 00000001)189vm-test-run-nixbot-gitlab> gitlab # [    0.002893] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS  BXPC     00000001 BXPC 00000001)190vm-test-run-nixbot-gitlab> gitlab # [    0.002895] ACPI: FACS 0x00000000BFFE0000 000040191vm-test-run-nixbot-gitlab> gitlab # [    0.002896] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS  BXPC     00000001 BXPC 00000001)192vm-test-run-nixbot-gitlab> gitlab # [    0.002898] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS  BXPC     00000001 BXPC 00000001)193vm-test-run-nixbot-gitlab> gitlab # [    0.002899] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS  BXPC     00000001 BXPC 00000001)194vm-test-run-nixbot-gitlab> gitlab # [    0.002900] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2]195vm-test-run-nixbot-gitlab> gitlab # [    0.002902] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e]196vm-test-run-nixbot-gitlab> gitlab # [    0.002902] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f]197vm-test-run-nixbot-gitlab> gitlab # [    0.002903] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482]198vm-test-run-nixbot-gitlab> gitlab # [    0.002903] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba]199vm-test-run-nixbot-gitlab> gitlab # [    0.002904] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2]200vm-test-run-nixbot-gitlab> gitlab # [    0.003404] No NUMA configuration found201vm-test-run-nixbot-gitlab> gitlab # [    0.003406] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff]202vm-test-run-nixbot-gitlab> gitlab # [    0.003409] NODE_DATA(0) allocated [mem 0x1bfffa780-0x1bffffcff]203vm-test-run-nixbot-gitlab> gitlab # [    0.006080] Zone ranges:204vm-test-run-nixbot-gitlab> gitlab # [    0.006081]   DMA      [mem 0x0000000000001000-0x0000000000ffffff]205vm-test-run-nixbot-gitlab> gitlab # [    0.006082]   DMA32    [mem 0x0000000001000000-0x00000000ffffffff]206vm-test-run-nixbot-gitlab> gitlab # [    0.006084]   Normal   [mem 0x0000000100000000-0x00000001bfffffff]207vm-test-run-nixbot-gitlab> gitlab # [    0.006085]   Device   empty208vm-test-run-nixbot-gitlab> gitlab # [    0.006085] Movable zone start for each node209vm-test-run-nixbot-gitlab> gitlab # [    0.006086] Early memory node ranges210vm-test-run-nixbot-gitlab> gitlab # [    0.006087]   node   0: [mem 0x0000000000001000-0x000000000009efff]211vm-test-run-nixbot-gitlab> gitlab # [    0.006088]   node   0: [mem 0x0000000000100000-0x00000000bffd7fff]212vm-test-run-nixbot-gitlab> gitlab # [    0.006088]   node   0: [mem 0x0000000100000000-0x00000001bfffffff]213vm-test-run-nixbot-gitlab> gitlab # [    0.006090] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff]214vm-test-run-nixbot-gitlab> gitlab # [    0.006110] On node 0, zone DMA: 1 pages in unavailable ranges215vm-test-run-nixbot-gitlab> gitlab # [    0.006389] On node 0, zone DMA: 97 pages in unavailable ranges216vm-test-run-nixbot-gitlab> gitlab # [    0.118479] On node 0, zone Normal: 40 pages in unavailable ranges217vm-test-run-nixbot-gitlab> gitlab # [    0.119530] ACPI: PM-Timer IO Port: 0x608218vm-test-run-nixbot-gitlab> gitlab # [    0.119547] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])219vm-test-run-nixbot-gitlab> gitlab # [    0.119586] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23220vm-test-run-nixbot-gitlab> gitlab # [    0.119588] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)221vm-test-run-nixbot-gitlab> gitlab # [    0.119590] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)222vm-test-run-nixbot-gitlab> gitlab # [    0.119592] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)223vm-test-run-nixbot-gitlab> gitlab # [    0.119593] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)224vm-test-run-nixbot-gitlab> gitlab # [    0.119594] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)225vm-test-run-nixbot-gitlab> gitlab # [    0.119596] ACPI: Using ACPI (MADT) for SMP configuration information226vm-test-run-nixbot-gitlab> gitlab # [    0.119597] ACPI: HPET id: 0x8086a201 base: 0xfed00000227vm-test-run-nixbot-gitlab> gitlab # [    0.119604] TSC deadline timer available228vm-test-run-nixbot-gitlab> gitlab # [    0.119609] CPU topo: Max. logical packages:   1229vm-test-run-nixbot-gitlab> gitlab # [    0.119609] CPU topo: Max. logical dies:       1230vm-test-run-nixbot-gitlab> gitlab # [    0.119610] CPU topo: Max. dies per package:   1231vm-test-run-nixbot-gitlab> gitlab # [    0.119613] CPU topo: Max. threads per core:   1232vm-test-run-nixbot-gitlab> gitlab # [    0.119614] CPU topo: Num. cores per package:     4233vm-test-run-nixbot-gitlab> gitlab # [    0.119614] CPU topo: Num. threads per package:   4234vm-test-run-nixbot-gitlab> gitlab # [    0.119614] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs235vm-test-run-nixbot-gitlab> gitlab # [    0.119641] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()236vm-test-run-nixbot-gitlab> gitlab # [    0.119661] kvm-guest: KVM setup pv remote TLB flush237vm-test-run-nixbot-gitlab> gitlab # [    0.119664] kvm-guest: setup PV sched yield238vm-test-run-nixbot-gitlab> gitlab # [    0.119684] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]239vm-test-run-nixbot-gitlab> gitlab # [    0.119686] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]240vm-test-run-nixbot-gitlab> gitlab # [    0.119687] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff]241vm-test-run-nixbot-gitlab> gitlab # [    0.119688] [mem 0xc0000000-0xfeffbfff] available for PCI devices242vm-test-run-nixbot-gitlab> gitlab # [    0.119690] Booting paravirtualized kernel on KVM243vm-test-run-nixbot-gitlab> gitlab # [    0.119694] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns244vm-test-run-nixbot-gitlab> gitlab # [    0.124149] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1245vm-test-run-nixbot-gitlab> gitlab # [    0.126469] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288246vm-test-run-nixbot-gitlab> gitlab # [    0.126520] kvm-guest: PV spinlocks enabled247vm-test-run-nixbot-gitlab> gitlab # [    0.126521] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear)248vm-test-run-nixbot-gitlab> gitlab # [    0.126525] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/igb97j0wm8q7h3ykz6g8nlk205nm0n7q-nixos-system-gitlab-test/init regInfo=/nix/store/zhf5qch3k3702akbp2aks0v6is94rga1-closure-info/registration console=ttyS0,115200n8 console=tty0249vm-test-run-nixbot-gitlab> gitlab # [    0.126623] Unknown kernel command line parameters "regInfo=/nix/store/zhf5qch3k3702akbp2aks0v6is94rga1-closure-info/registration", will be passed to user space.250vm-test-run-nixbot-gitlab> gitlab # [    0.126636] random: crng init done251vm-test-run-nixbot-gitlab> gitlab # [    0.126637] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes252vm-test-run-nixbot-gitlab> gitlab # [    0.135877] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)253vm-test-run-nixbot-gitlab> gitlab # [    0.140593] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)254vm-test-run-nixbot-gitlab> gitlab # [    0.140679] software IO TLB: area num 4.255vm-test-run-nixbot-gitlab> gitlab # [    0.218194] Fallback order for Node 0: 0256vm-test-run-nixbot-gitlab> gitlab # [    0.218202] Built 1 zonelists, mobility grouping on.  Total pages: 1572726257vm-test-run-nixbot-gitlab> gitlab # [    0.218205] Policy zone: Normal258vm-test-run-nixbot-gitlab> gitlab # [    0.220772] mem auto-init: stack:all(zero), heap alloc:on, heap free:off259vm-test-run-nixbot-gitlab> gitlab # [    0.231116] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1260vm-test-run-nixbot-gitlab> gitlab # [    0.244804] allocated 12582912 bytes of page_ext261vm-test-run-nixbot-gitlab> gitlab # [    0.254941] ftrace: allocating 48565 entries in 192 pages262vm-test-run-nixbot-gitlab> gitlab # [    0.254944] ftrace: allocated 192 pages with 2 groups263vm-test-run-nixbot-gitlab> gitlab # [    0.255795] Dynamic Preempt: lazy264vm-test-run-nixbot-gitlab> gitlab # [    0.255974] rcu: Preemptible hierarchical RCU implementation.265vm-test-run-nixbot-gitlab> gitlab # [    0.255974] rcu: 	RCU event tracing is enabled.266vm-test-run-nixbot-gitlab> gitlab # [    0.255975] rcu: 	RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.267vm-test-run-nixbot-gitlab> gitlab # [    0.255976] 	Trampoline variant of Tasks RCU enabled.268vm-test-run-nixbot-gitlab> gitlab # [    0.255977] 	Rude variant of Tasks RCU enabled.269vm-test-run-nixbot-gitlab> gitlab # [    0.255977] 	Tracing variant of Tasks RCU enabled.270vm-test-run-nixbot-gitlab> gitlab # [    0.255978] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.271vm-test-run-nixbot-gitlab> gitlab # [    0.255979] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4272vm-test-run-nixbot-gitlab> gitlab # [    0.255998] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.273vm-test-run-nixbot-gitlab> gitlab # [    0.256000] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.274vm-test-run-nixbot-gitlab> gitlab # [    0.256001] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.275vm-test-run-nixbot-gitlab> gitlab # [    0.260333] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16276vm-test-run-nixbot-gitlab> gitlab # [    0.260639] rcu: srcu_init: Setting srcu_struct sizes based on contention.277vm-test-run-nixbot-gitlab> gitlab # [    0.260769] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)278vm-test-run-nixbot-gitlab> gitlab # [    0.268200] Console: colour VGA+ 80x25279vm-test-run-nixbot-gitlab> gitlab # [    0.268204] printk: legacy console [tty0] enabled280vm-test-run-nixbot-gitlab> gitlab # [    0.312140] printk: legacy console [ttyS0] enabled281vm-test-run-nixbot-gitlab> gitlab # [    0.513887] ACPI: Core revision 20250807282vm-test-run-nixbot-gitlab> gitlab # [    0.515542] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns283vm-test-run-nixbot-gitlab> gitlab # [    0.518381] APIC: Switch to symmetric I/O mode setup284vm-test-run-nixbot-gitlab> gitlab # [    0.520147] x2apic enabled285vm-test-run-nixbot-gitlab> gitlab # [    0.521397] APIC: Switched APIC routing to: physical x2apic286vm-test-run-nixbot-gitlab> gitlab # [    0.523118] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask()287vm-test-run-nixbot-gitlab> gitlab # [    0.525352] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself()288vm-test-run-nixbot-gitlab> gitlab # [    0.528121] kvm-guest: setup PV IPIs289vm-test-run-nixbot-gitlab> gitlab # [    0.530509] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1290vm-test-run-nixbot-gitlab> gitlab # [    0.532347] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229840d1729, max_idle_ns: 440795276167 ns291vm-test-run-nixbot-gitlab> gitlab # [    0.535521] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400010)292vm-test-run-nixbot-gitlab> gitlab # [    0.536637] x86/cpu: User Mode Instruction Prevention (UMIP) activated293vm-test-run-nixbot-gitlab> gitlab # [    0.538679] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127294vm-test-run-nixbot-gitlab> gitlab # [    0.540519] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0295vm-test-run-nixbot-gitlab> gitlab # [    0.542524] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto296vm-test-run-nixbot-gitlab> gitlab # [    0.544519] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl297vm-test-run-nixbot-gitlab> gitlab # [    0.545519] Transient Scheduler Attacks: Mitigation: Clear CPU buffers298vm-test-run-nixbot-gitlab> gitlab # [    0.547519] Spectre V2 : Mitigation: Enhanced / Automatic IBRS299vm-test-run-nixbot-gitlab> gitlab # [    0.549519] Speculative Return Stack Overflow: Mitigation: Safe RET300vm-test-run-nixbot-gitlab> gitlab # [    0.550519] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization301vm-test-run-nixbot-gitlab> gitlab # [    0.551525] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier302vm-test-run-nixbot-gitlab> gitlab # [    0.552520] active return thunk: srso_alias_return_thunk303vm-test-run-nixbot-gitlab> gitlab # [    0.553547] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'304vm-test-run-nixbot-gitlab> gitlab # [    0.554519] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'305vm-test-run-nixbot-gitlab> gitlab # [    0.555519] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'306vm-test-run-nixbot-gitlab> gitlab # [    0.556519] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'307vm-test-run-nixbot-gitlab> gitlab # [    0.557519] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'308vm-test-run-nixbot-gitlab> gitlab # [    0.558519] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'309vm-test-run-nixbot-gitlab> gitlab # [    0.559519] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'310vm-test-run-nixbot-gitlab> gitlab # [    0.560519] x86/fpu: Supporting XSAVE feature 0x800: 'Control-flow User registers'311vm-test-run-nixbot-gitlab> gitlab # [    0.561519] x86/fpu: Supporting XSAVE feature 0x1000: 'Control-flow Kernel registers (KVM only)'312vm-test-run-nixbot-gitlab> gitlab # [    0.563519] x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256313vm-test-run-nixbot-gitlab> gitlab # [    0.564519] x86/fpu: xstate_offset[5]:  832, xstate_sizes[5]:   64314vm-test-run-nixbot-gitlab> gitlab # [    0.565519] x86/fpu: xstate_offset[6]:  896, xstate_sizes[6]:  512315vm-test-run-nixbot-gitlab> gitlab # [    0.566519] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024316vm-test-run-nixbot-gitlab> gitlab # [    0.568519] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]:    8317vm-test-run-nixbot-gitlab> gitlab # [    0.570519] x86/fpu: xstate_offset[11]: 2440, xstate_sizes[11]:   16318vm-test-run-nixbot-gitlab> gitlab # [    0.571519] x86/fpu: xstate_offset[12]: 2456, xstate_sizes[12]:   24319vm-test-run-nixbot-gitlab> gitlab # [    0.572519] x86/fpu: Enabled xstate features 0x1ae7, context size is 2480 bytes, using 'compacted' format.320vm-test-run-nixbot-gitlab> gitlab # [    0.602230] Freeing SMP alternatives memory: 44K321vm-test-run-nixbot-gitlab> gitlab # [    0.603521] pid_max: default: 32768 minimum: 301322vm-test-run-nixbot-gitlab> gitlab # [    0.604639] LSM: initializing lsm=capability,landlock,yama,bpf,ima323vm-test-run-nixbot-gitlab> gitlab # [    0.606640] landlock: Up and running.324vm-test-run-nixbot-gitlab> gitlab # [    0.608393] Yama: becoming mindful.325vm-test-run-nixbot-gitlab> gitlab # [    0.609732] LSM support for eBPF active326vm-test-run-nixbot-gitlab> gitlab # [    0.610745] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)327vm-test-run-nixbot-gitlab> gitlab # [    0.612669] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)328vm-test-run-nixbot-gitlab> gitlab # [    0.615341] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)329vm-test-run-nixbot-gitlab> gitlab # [    0.616100] Performance Events: Fam17h+ core perfctr, AMD PMU driver.330vm-test-run-nixbot-gitlab> gitlab # [    0.616524] ... version:                   2331vm-test-run-nixbot-gitlab> gitlab # [    0.617520] ... bit width:                 48332vm-test-run-nixbot-gitlab> gitlab # [    0.618576] ... generic counters:          6333vm-test-run-nixbot-gitlab> gitlab # [    0.619521] ... generic bitmap:            000000000000003f334vm-test-run-nixbot-gitlab> gitlab # [    0.620521] ... fixed-purpose counters:    0335vm-test-run-nixbot-gitlab> gitlab # [    0.621520] ... fixed-purpose bitmap:      0000000000000000336vm-test-run-nixbot-gitlab> gitlab # [    0.622521] ... value mask:                0000ffffffffffff337vm-test-run-nixbot-gitlab> gitlab # [    0.623521] ... max period:                00007fffffffffff338vm-test-run-nixbot-gitlab> gitlab # [    0.624520] ... global_ctrl mask:          000000000000003f339vm-test-run-nixbot-gitlab> gitlab # [    0.625646] signal: max sigframe size: 3376340vm-test-run-nixbot-gitlab> gitlab # [    0.626620] rcu: Hierarchical SRCU implementation.341vm-test-run-nixbot-gitlab> gitlab # [    0.627525] rcu: 	Max phase no-delay instances is 400.342vm-test-run-nixbot-gitlab> gitlab # [    0.628720] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level343vm-test-run-nixbot-gitlab> gitlab # [    0.634098] smp: Bringing up secondary CPUs ...344vm-test-run-nixbot-gitlab> gitlab # [    0.634942] smpboot: x86: Booting SMP configuration:345vm-test-run-nixbot-gitlab> gitlab # [    0.635531] .... node  #0, CPUs:      #1 #2 #3346vm-test-run-nixbot-gitlab> gitlab # [    0.639606] smp: Brought up 1 node, 4 CPUs347vm-test-run-nixbot-gitlab> gitlab # [    0.641521] smpboot: Total of 4 processors activated (19200.08 BogoMIPS)348vm-test-run-nixbot-gitlab> gitlab # [    0.643085] Memory: 6013144K/6290904K available (17098K kernel code, 2721K rwdata, 13520K rodata, 3644K init, 3008K bss, 258296K reserved, 0K cma-reserved)349vm-test-run-nixbot-gitlab> gitlab # [    0.644969] devtmpfs: initialized350vm-test-run-nixbot-gitlab> gitlab # [    0.646579] x86/mm: Memory block size: 128MB351vm-test-run-nixbot-gitlab> gitlab # [    0.650532] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns352vm-test-run-nixbot-gitlab> gitlab # [    0.651574] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)353vm-test-run-nixbot-gitlab> gitlab # [    0.652601] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).354vm-test-run-nixbot-gitlab> gitlab # [    0.653650] pinctrl core: initialized pinctrl subsystem355vm-test-run-nixbot-gitlab> gitlab # [    0.654967] PM: RTC time: 12:35:41, date: 2026-07-04356vm-test-run-nixbot-gitlab> gitlab # [    0.658667] NET: Registered PF_NETLINK/PF_ROUTE protocol family357vm-test-run-nixbot-gitlab> gitlab # [    0.660858] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations358vm-test-run-nixbot-gitlab> gitlab # [    0.661599] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations359vm-test-run-nixbot-gitlab> gitlab # [    0.663611] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations360vm-test-run-nixbot-gitlab> gitlab # [    0.664536] audit: initializing netlink subsys (disabled)361vm-test-run-nixbot-gitlab> gitlab # [    0.665713] audit: type=2000 audit(1783168541.657:1): state=initialized audit_enabled=0 res=1362vm-test-run-nixbot-gitlab> gitlab # [    0.665750] thermal_sys: Registered thermal governor 'fair_share'363vm-test-run-nixbot-gitlab> gitlab # [    0.667522] thermal_sys: Registered thermal governor 'bang_bang'364vm-test-run-nixbot-gitlab> gitlab # [    0.668521] thermal_sys: Registered thermal governor 'step_wise'365vm-test-run-nixbot-gitlab> gitlab # [    0.669521] thermal_sys: Registered thermal governor 'user_space'366vm-test-run-nixbot-gitlab> gitlab # [    0.670521] thermal_sys: Registered thermal governor 'power_allocator'367vm-test-run-nixbot-gitlab> gitlab # [    0.671547] cpuidle: using governor menu368vm-test-run-nixbot-gitlab> gitlab # [    0.674761] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5369vm-test-run-nixbot-gitlab> gitlab # [    0.675921] PCI: Using configuration type 1 for base access370vm-test-run-nixbot-gitlab> gitlab # [    0.676521] PCI: Using configuration type 1 for extended access371vm-test-run-nixbot-gitlab> gitlab # [    0.677706] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.372vm-test-run-nixbot-gitlab> gitlab # [    0.680389] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages373vm-test-run-nixbot-gitlab> gitlab # [    0.685534] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page374vm-test-run-nixbot-gitlab> gitlab # [    0.689531] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages375vm-test-run-nixbot-gitlab> gitlab # [    0.694529] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page376vm-test-run-nixbot-gitlab> gitlab # [    0.705078] ACPI: Added _OSI(Module Device)377vm-test-run-nixbot-gitlab> gitlab # [    0.708533] ACPI: Added _OSI(Processor Device)378vm-test-run-nixbot-gitlab> gitlab # [    0.711530] ACPI: Added _OSI(Processor Aggregator Device)379vm-test-run-nixbot-gitlab> gitlab # [    0.720651] ACPI: 1 ACPI AML tables successfully acquired and loaded380vm-test-run-nixbot-gitlab> gitlab # [    0.728825] ACPI: Interpreter enabled381vm-test-run-nixbot-gitlab> gitlab # [    0.730610] ACPI: PM: (supports S0 S3 S4 S5)382vm-test-run-nixbot-gitlab> gitlab # [    0.731523] ACPI: Using IOAPIC for interrupt routing383vm-test-run-nixbot-gitlab> gitlab # [    0.733644] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug384vm-test-run-nixbot-gitlab> gitlab # [    0.735522] PCI: Using E820 reservations for host bridge windows385vm-test-run-nixbot-gitlab> gitlab # [    0.737782] ACPI: Enabled 2 GPEs in block 00 to 0F386vm-test-run-nixbot-gitlab> gitlab # [    0.744758] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])387vm-test-run-nixbot-gitlab> gitlab # [    0.746530] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]388vm-test-run-nixbot-gitlab> gitlab # [    0.750162] acpiphp: Slot [3] registered389vm-test-run-nixbot-gitlab> gitlab # [    0.750565] acpiphp: Slot [4] registered390vm-test-run-nixbot-gitlab> gitlab # [    0.752562] acpiphp: Slot [5] registered391vm-test-run-nixbot-gitlab> gitlab # [    0.753580] acpiphp: Slot [6] registered392vm-test-run-nixbot-gitlab> gitlab # [    0.754562] acpiphp: Slot [7] registered393vm-test-run-nixbot-gitlab> gitlab # [    0.756564] acpiphp: Slot [8] registered394vm-test-run-nixbot-gitlab> gitlab # [    0.757590] acpiphp: Slot [9] registered395vm-test-run-nixbot-gitlab> gitlab # [    0.758564] acpiphp: Slot [10] registered396vm-test-run-nixbot-gitlab> gitlab # [    0.760574] acpiphp: Slot [11] registered397vm-test-run-nixbot-gitlab> gitlab # [    0.761563] acpiphp: Slot [12] registered398vm-test-run-nixbot-gitlab> gitlab # [    0.762580] acpiphp: Slot [13] registered399vm-test-run-nixbot-gitlab> gitlab # [    0.763557] acpiphp: Slot [14] registered400vm-test-run-nixbot-gitlab> gitlab # [    0.765555] acpiphp: Slot [15] registered401vm-test-run-nixbot-gitlab> gitlab # [    0.766554] acpiphp: Slot [16] registered402vm-test-run-nixbot-gitlab> gitlab # [    0.767572] acpiphp: Slot [17] registered403vm-test-run-nixbot-gitlab> gitlab # [    0.769556] acpiphp: Slot [18] registered404vm-test-run-nixbot-gitlab> gitlab # [    0.770555] acpiphp: Slot [19] registered405vm-test-run-nixbot-gitlab> gitlab # [    0.771555] acpiphp: Slot [20] registered406vm-test-run-nixbot-gitlab> gitlab # [    0.773571] acpiphp: Slot [21] registered407vm-test-run-nixbot-gitlab> gitlab # [    0.774558] acpiphp: Slot [22] registered408vm-test-run-nixbot-gitlab> gitlab # [    0.775556] acpiphp: Slot [23] registered409vm-test-run-nixbot-gitlab> gitlab # [    0.776575] acpiphp: Slot [24] registered410vm-test-run-nixbot-gitlab> gitlab # [    0.778556] acpiphp: Slot [25] registered411vm-test-run-nixbot-gitlab> gitlab # [    0.779555] acpiphp: Slot [26] registered412vm-test-run-nixbot-gitlab> gitlab # [    0.780555] acpiphp: Slot [27] registered413vm-test-run-nixbot-gitlab> gitlab # [    0.782570] acpiphp: Slot [28] registered414vm-test-run-nixbot-gitlab> gitlab # [    0.783556] acpiphp: Slot [29] registered415vm-test-run-nixbot-gitlab> gitlab # [    0.784557] acpiphp: Slot [30] registered416vm-test-run-nixbot-gitlab> gitlab # [    0.785556] acpiphp: Slot [31] registered417vm-test-run-nixbot-gitlab> gitlab # [    0.787570] PCI host bridge to bus 0000:00418vm-test-run-nixbot-gitlab> gitlab # [    0.788530] pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7 window]419vm-test-run-nixbot-gitlab> gitlab # [    0.790522] pci_bus 0000:00: root bus resource [io  0x0d00-0xffff window]420vm-test-run-nixbot-gitlab> gitlab # [    0.792522] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]421vm-test-run-nixbot-gitlab> gitlab # [    0.794523] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]422vm-test-run-nixbot-gitlab> gitlab # [    0.796523] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window]423vm-test-run-nixbot-gitlab> gitlab # [    0.799525] pci_bus 0000:00: root bus resource [bus 00-ff]424vm-test-run-nixbot-gitlab> gitlab # [    0.801036] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint425vm-test-run-nixbot-gitlab> gitlab # [    0.804503] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint426vm-test-run-nixbot-gitlab> gitlab # [    0.808449] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint427vm-test-run-nixbot-gitlab> gitlab # [    0.813610] pci 0000:00:01.1: BAR 4 [io  0xc220-0xc22f]428vm-test-run-nixbot-gitlab> gitlab # [    0.815588] pci 0000:00:01.1: BAR 0 [io  0x01f0-0x01f7]: legacy IDE quirk429vm-test-run-nixbot-gitlab> gitlab # [    0.817521] pci 0000:00:01.1: BAR 1 [io  0x03f6]: legacy IDE quirk430vm-test-run-nixbot-gitlab> gitlab # [    0.818522] pci 0000:00:01.1: BAR 2 [io  0x0170-0x0177]: legacy IDE quirk431vm-test-run-nixbot-gitlab> gitlab # [    0.820522] pci 0000:00:01.1: BAR 3 [io  0x0376]: legacy IDE quirk432vm-test-run-nixbot-gitlab> gitlab # [    0.822855] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint433vm-test-run-nixbot-gitlab> gitlab # [    0.827276] pci 0000:00:01.2: BAR 4 [io  0xc140-0xc15f]434vm-test-run-nixbot-gitlab> gitlab # [    0.830860] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint435vm-test-run-nixbot-gitlab> gitlab # [    0.833201] pci 0000:00:01.3: quirk: [io  0x0600-0x063f] claimed by PIIX4 ACPI436vm-test-run-nixbot-gitlab> gitlab # [    0.835536] pci 0000:00:01.3: quirk: [io  0x0700-0x070f] claimed by PIIX4 SMB437vm-test-run-nixbot-gitlab> gitlab # [    0.837973] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint438vm-test-run-nixbot-gitlab> gitlab # [    0.847605] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]439vm-test-run-nixbot-gitlab> gitlab # [    0.849570] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]440vm-test-run-nixbot-gitlab> gitlab # [    0.851646] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref]441vm-test-run-nixbot-gitlab> gitlab # [    0.857185] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]442vm-test-run-nixbot-gitlab> gitlab # [    0.892546] pci 0000:00:02.0: pci_fixup_video+0x0/0x100 took 35156 usecs443vm-test-run-nixbot-gitlab> gitlab # [    0.900535] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint444vm-test-run-nixbot-gitlab> gitlab # [    0.912564] pci 0000:00:03.0: BAR 0 [io  0xc160-0xc17f]445vm-test-run-nixbot-gitlab> gitlab # [    0.916572] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]446vm-test-run-nixbot-gitlab> gitlab # [    0.918578] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref]447vm-test-run-nixbot-gitlab> gitlab # [    0.920538] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]448vm-test-run-nixbot-gitlab> gitlab # [    0.925058] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint449vm-test-run-nixbot-gitlab> gitlab # [    0.930560] pci 0000:00:04.0: BAR 0 [io  0xc180-0xc19f]450vm-test-run-nixbot-gitlab> gitlab # [    0.932544] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]451vm-test-run-nixbot-gitlab> gitlab # [    0.933576] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref]452vm-test-run-nixbot-gitlab> gitlab # [    0.938768] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint453vm-test-run-nixbot-gitlab> gitlab # [    0.943560] pci 0000:00:05.0: BAR 0 [io  0xc1a0-0xc1bf]454vm-test-run-nixbot-gitlab> gitlab # [    0.945536] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]455vm-test-run-nixbot-gitlab> gitlab # [    0.947575] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref]456vm-test-run-nixbot-gitlab> gitlab # [    0.952097] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint457vm-test-run-nixbot-gitlab> gitlab # [    0.956558] pci 0000:00:06.0: BAR 0 [io  0xc1c0-0xc1df]458vm-test-run-nixbot-gitlab> gitlab # [    0.958537] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]459vm-test-run-nixbot-gitlab> gitlab # [    0.960597] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref]460vm-test-run-nixbot-gitlab> gitlab # [    0.964574] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint461vm-test-run-nixbot-gitlab> gitlab # [    0.969570] pci 0000:00:07.0: BAR 0 [io  0xc000-0xc07f]462vm-test-run-nixbot-gitlab> gitlab # [    0.971538] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]463vm-test-run-nixbot-gitlab> gitlab # [    0.973576] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref]464vm-test-run-nixbot-gitlab> gitlab # [    0.977600] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint465vm-test-run-nixbot-gitlab> gitlab # [    0.984567] pci 0000:00:08.0: BAR 0 [io  0xc080-0xc0ff]466vm-test-run-nixbot-gitlab> gitlab # [    0.986541] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]467vm-test-run-nixbot-gitlab> gitlab # [    0.989646] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref]468vm-test-run-nixbot-gitlab> gitlab # [    0.997840] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint469vm-test-run-nixbot-gitlab> gitlab # [    1.004542] pci 0000:00:09.0: BAR 0 [io  0xc1e0-0xc1ff]470vm-test-run-nixbot-gitlab> gitlab # [    1.006538] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]471vm-test-run-nixbot-gitlab> gitlab # [    1.008576] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref]472vm-test-run-nixbot-gitlab> gitlab # [    1.010537] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref]473vm-test-run-nixbot-gitlab> gitlab # [    1.014701] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint474vm-test-run-nixbot-gitlab> gitlab # [    1.019579] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]475vm-test-run-nixbot-gitlab> gitlab # [    1.021575] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref]476vm-test-run-nixbot-gitlab> gitlab # [    1.025587] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint477vm-test-run-nixbot-gitlab> gitlab # [    1.031570] pci 0000:00:0b.0: BAR 0 [io  0xc100-0xc13f]478vm-test-run-nixbot-gitlab> gitlab # [    1.033544] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]479vm-test-run-nixbot-gitlab> gitlab # [    1.035600] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref]480vm-test-run-nixbot-gitlab> gitlab # [    1.039664] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint481vm-test-run-nixbot-gitlab> gitlab # [    1.044545] pci 0000:00:0c.0: BAR 0 [io  0xc200-0xc21f]482vm-test-run-nixbot-gitlab> gitlab # [    1.045536] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff]483vm-test-run-nixbot-gitlab> gitlab # [    1.047576] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref]484vm-test-run-nixbot-gitlab> gitlab # [    1.070031] ACPI: PCI: Interrupt link LNKA configured for IRQ 10485vm-test-run-nixbot-gitlab> gitlab # [    1.075069] ACPI: PCI: Interrupt link LNKB configured for IRQ 10486vm-test-run-nixbot-gitlab> gitlab # [    1.080040] ACPI: PCI: Interrupt link LNKC configured for IRQ 11487vm-test-run-nixbot-gitlab> gitlab # [    1.085049] ACPI: PCI: Interrupt link LNKD configured for IRQ 11488vm-test-run-nixbot-gitlab> gitlab # [    1.089828] ACPI: PCI: Interrupt link LNKS configured for IRQ 9489vm-test-run-nixbot-gitlab> gitlab # [    1.097579] iommu: Default domain type: Translated490vm-test-run-nixbot-gitlab> gitlab # [    1.100547] iommu: DMA domain TLB invalidation policy: lazy mode491vm-test-run-nixbot-gitlab> gitlab # [    1.103927] ACPI: bus type USB registered492vm-test-run-nixbot-gitlab> gitlab # [    1.105652] usbcore: registered new interface driver usbfs493vm-test-run-nixbot-gitlab> gitlab # [    1.106548] usbcore: registered new interface driver hub494vm-test-run-nixbot-gitlab> gitlab # [    1.108545] usbcore: registered new device driver usb495vm-test-run-nixbot-gitlab> gitlab # [    1.111187] NetLabel: Initializing496vm-test-run-nixbot-gitlab> gitlab # [    1.112522] NetLabel:  domain hash size = 128497vm-test-run-nixbot-gitlab> gitlab # [    1.114522] NetLabel:  protocols = UNLABELED CIPSOv4 CALIPSO498vm-test-run-nixbot-gitlab> gitlab # [    1.115622] NetLabel:  unlabeled traffic allowed by default499vm-test-run-nixbot-gitlab> gitlab # [    1.117538] PCI: Using ACPI for IRQ routing500vm-test-run-nixbot-gitlab> gitlab # [    1.119991] pci 0000:00:02.0: vgaarb: setting as boot VGA device501vm-test-run-nixbot-gitlab> gitlab # [    1.120526] pci 0000:00:02.0: vgaarb: bridge control possible502vm-test-run-nixbot-gitlab> gitlab # [    1.120526] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none503vm-test-run-nixbot-gitlab> gitlab # [    1.125528] vgaarb: loaded504vm-test-run-nixbot-gitlab> gitlab # [    1.126751] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0505vm-test-run-nixbot-gitlab> gitlab # [    1.128522] hpet0: 3 comparators, 64-bit 100.000000 MHz counter506vm-test-run-nixbot-gitlab> gitlab # [    1.133641] clocksource: Switched to clocksource kvm-clock507vm-test-run-nixbot-gitlab> gitlab # [    1.140164] VFS: Disk quotas dquot_6.6.0508vm-test-run-nixbot-gitlab> gitlab # [    1.143636] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)509vm-test-run-nixbot-gitlab> gitlab # [    1.150450] pnp: PnP ACPI init510vm-test-run-nixbot-gitlab> gitlab # [    1.154905] pnp: PnP ACPI: found 6 devices511vm-test-run-nixbot-gitlab> gitlab # [    1.171601] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns512vm-test-run-nixbot-gitlab> gitlab # [    1.178260] clocksource: Switched to clocksource acpi_pm513vm-test-run-nixbot-gitlab> gitlab # [    1.182828] NET: Registered PF_INET protocol family514vm-test-run-nixbot-gitlab> gitlab # [    1.190129] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)515vm-test-run-nixbot-gitlab> gitlab # [    1.220595] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)516vm-test-run-nixbot-gitlab> gitlab # [    1.223279] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)517vm-test-run-nixbot-gitlab> gitlab # [    1.225722] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)518vm-test-run-nixbot-gitlab> gitlab # [    1.230641] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)519vm-test-run-nixbot-gitlab> gitlab # [    1.233007] TCP: Hash tables configured (established 65536 bind 65536)520vm-test-run-nixbot-gitlab> gitlab # [    1.235800] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)521vm-test-run-nixbot-gitlab> gitlab # [    1.243345] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)522vm-test-run-nixbot-gitlab> gitlab # [    1.250060] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)523vm-test-run-nixbot-gitlab> gitlab # [    1.258512] NET: Registered PF_UNIX/PF_LOCAL protocol family524vm-test-run-nixbot-gitlab> gitlab # [    1.262348] NET: Registered PF_XDP protocol family525vm-test-run-nixbot-gitlab> gitlab # [    1.263956] pci_bus 0000:00: resource 4 [io  0x0000-0x0cf7 window]526vm-test-run-nixbot-gitlab> gitlab # [    1.265988] pci_bus 0000:00: resource 5 [io  0x0d00-0xffff window]527vm-test-run-nixbot-gitlab> gitlab # [    1.267906] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]528vm-test-run-nixbot-gitlab> gitlab # [    1.270049] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window]529vm-test-run-nixbot-gitlab> gitlab # [    1.272163] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window]530vm-test-run-nixbot-gitlab> gitlab # [    1.274585] pci 0000:00:01.0: PIIX3: Enabling Passive Release531vm-test-run-nixbot-gitlab> gitlab # [    1.276475] pci 0000:00:00.0: Limiting direct PCI/PCI transfers532vm-test-run-nixbot-gitlab> gitlab # [    1.280067] ACPI: \_SB_.LNKD: Enabled at IRQ 11533vm-test-run-nixbot-gitlab> gitlab # [    1.283666] PCI: CLS 0 bytes, default 64534vm-test-run-nixbot-gitlab> gitlab # [    1.285144] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)535vm-test-run-nixbot-gitlab> gitlab # [    1.285673] Trying to unpack rootfs image as initramfs...536vm-test-run-nixbot-gitlab> gitlab # [    1.287088] software IO TLB: mapped [mem 0x00000000ba4e0000-0x00000000be4e0000] (64MB)537vm-test-run-nixbot-gitlab> gitlab # [    1.287254] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229840d1729, max_idle_ns: 440795276167 ns538vm-test-run-nixbot-gitlab> gitlab # [    1.317153] Initialise system trusted keyrings539vm-test-run-nixbot-gitlab> gitlab # [    1.319389] workingset: timestamp_bits=40 max_order=21 bucket_order=0540vm-test-run-nixbot-gitlab> gitlab # [    1.333279] Key type asymmetric registered541vm-test-run-nixbot-gitlab> gitlab # [    1.334712] Asymmetric key parser 'x509' registered542vm-test-run-nixbot-gitlab> gitlab # [    1.336405] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)543vm-test-run-nixbot-gitlab> gitlab # [    1.339394] io scheduler mq-deadline registered544vm-test-run-nixbot-gitlab> gitlab # [    1.340932] io scheduler kyber registered545vm-test-run-nixbot-gitlab> gitlab # [    1.345493] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled546vm-test-run-nixbot-gitlab> gitlab # [    1.347985] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A547vm-test-run-nixbot-gitlab> gitlab # [    1.352156] Linux agpgart interface v0.103548vm-test-run-nixbot-gitlab> gitlab # [    1.353606] ACPI: bus type drm_connector registered549vm-test-run-nixbot-gitlab> gitlab # [    1.357523] usbcore: registered new interface driver usbserial_generic550vm-test-run-nixbot-gitlab> gitlab # [    1.359585] usbserial: USB Serial support registered for generic551vm-test-run-nixbot-gitlab> gitlab # [    1.361463] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.552vm-test-run-nixbot-gitlab> gitlab # [    1.361463] Please enable it if your BIOS has the CPPC option.553vm-test-run-nixbot-gitlab> gitlab # [    1.365593] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled554vm-test-run-nixbot-gitlab> gitlab # [    1.368065] drop_monitor: Initializing network drop monitor service555vm-test-run-nixbot-gitlab> gitlab # [    1.370172] NET: Registered PF_INET6 protocol family556vm-test-run-nixbot-gitlab> gitlab # [    1.372857] Segment Routing with IPv6557vm-test-run-nixbot-gitlab> gitlab # [    1.374203] In-situ OAM (IOAM) with IPv6558vm-test-run-nixbot-gitlab> gitlab # [    1.378617] IPI shorthand broadcast: enabled559vm-test-run-nixbot-gitlab> gitlab # [    1.386535] sched_clock: Marking stable (1123062019, 262868650)->(1466181850, -80251181)560vm-test-run-nixbot-gitlab> gitlab # [    1.392376] registered taskstats version 1561vm-test-run-nixbot-gitlab> gitlab # [    1.395769] Loading compiled-in X.509 certificates562vm-test-run-nixbot-gitlab> gitlab # [    1.410483] Demotion targets for Node 0: null563vm-test-run-nixbot-gitlab> gitlab # [    1.412218] Key type .fscrypt registered564vm-test-run-nixbot-gitlab> gitlab # [    1.413497] Key type fscrypt-provisioning registered565vm-test-run-nixbot-gitlab> gitlab # [    1.415222] ima: No TPM chip found, activating TPM-bypass!566vm-test-run-nixbot-gitlab> gitlab # [    1.416942] ima: Allocated hash algorithm: sha1567vm-test-run-nixbot-gitlab> gitlab # [    1.418424] ima: No architecture policies found568vm-test-run-nixbot-gitlab> gitlab # [    1.420260] PM:   Magic number: 2:607:581569vm-test-run-nixbot-gitlab> gitlab # [    1.422659] RAS: Correctable Errors collector initialized.570vm-test-run-nixbot-gitlab> gitlab # [    1.428164] clk: Disabling unused clocks571vm-test-run-nixbot-gitlab> gitlab # [    1.429506] PM: genpd: Disabling unused power domains572vm-test-run-nixbot-gitlab> gitlab # [    1.601698] Freeing initrd memory: 27584K573vm-test-run-nixbot-gitlab> gitlab # [    1.610563] Freeing unused decrypted memory: 2028K574vm-test-run-nixbot-gitlab> gitlab # [    1.619563] Freeing unused kernel image (initmem) memory: 3644K575vm-test-run-nixbot-gitlab> gitlab # [    1.624354] Write protecting the kernel read-only data: 32768k576vm-test-run-nixbot-gitlab> gitlab # [    1.632099] Freeing unused kernel image (text/rodata gap) memory: 1332K577vm-test-run-nixbot-gitlab> gitlab # [    1.638585] Freeing unused kernel image (rodata/data gap) memory: 816K578vm-test-run-nixbot-gitlab> gitlab # [    1.702296] x86/mm: Checked W+X mappings: passed, no W+X pages found.579vm-test-run-nixbot-gitlab> gitlab # [    1.704377] Run /init as init process580vm-test-run-nixbot-gitlab> gitlab # [    1.719075] systemd[1]: Inserted module 'autofs4'581vm-test-run-nixbot-gitlab> gitlab # [    1.748975] fuse: init (API version 7.45)582vm-test-run-nixbot-gitlab> gitlab # [    1.759821] ACPI: \_SB_.LNKC: Enabled at IRQ 10583vm-test-run-nixbot-gitlab> gitlab # [    1.769597] ACPI: \_SB_.LNKA: Enabled at IRQ 10584vm-test-run-nixbot-gitlab> gitlab # [    1.774794] ACPI: \_SB_.LNKB: Enabled at IRQ 11585vm-test-run-nixbot-gitlab> gitlab # [    1.844216] systemd[1]: Successfully made /usr/ read-only.586vm-test-run-nixbot-gitlab> gitlab # [    2.189980] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)587vm-test-run-nixbot-gitlab> gitlab # [    2.214089] systemd[1]: Detected virtualization kvm.588vm-test-run-nixbot-gitlab> gitlab # [    2.218576] systemd[1]: Detected architecture x86-64.589vm-test-run-nixbot-gitlab> gitlab # [    2.223000] systemd[1]: Running in initrd.590vm-test-run-nixbot-gitlab> gitlab # [    2.227860] systemd[1]: Initializing machine ID from random generator.591vm-test-run-nixbot-gitlab> gitlab # [    2.233726] systemd[1]: Hostname set to <gitlab>.592vm-test-run-nixbot-gitlab> gitlab # [    2.323116] systemd[1]: Queued start job for default target Initrd Default Target.593vm-test-run-nixbot-gitlab> gitlab # [    2.504677] systemd[1]: Created slice Slice /system/modprobe.594vm-test-run-nixbot-gitlab> gitlab # [    2.510016] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.595vm-test-run-nixbot-gitlab> gitlab # [    2.516193] systemd[1]: Expecting device /dev/disk/by-label/nix-store...596vm-test-run-nixbot-gitlab> gitlab # [    2.521332] systemd[1]: Expecting device /dev/disk/by-label/nixos...597vm-test-run-nixbot-gitlab> gitlab # [    2.526367] systemd[1]: Reached target Path Units.598vm-test-run-nixbot-gitlab> gitlab # [    2.530360] systemd[1]: Reached target Slice Units.599vm-test-run-nixbot-gitlab> gitlab # [    2.534374] systemd[1]: Reached target Swaps.600vm-test-run-nixbot-gitlab> gitlab # [    2.537438] systemd[1]: Reached target Timer Units.601vm-test-run-nixbot-gitlab> gitlab # [    2.541018] systemd[1]: Listening on D-Bus System Message Bus Socket.602vm-test-run-nixbot-gitlab> gitlab # [    2.545350] systemd[1]: Listening on Journal Socket (/dev/log).603vm-test-run-nixbot-gitlab> gitlab # [    2.549405] systemd[1]: Listening on Journal Sockets.604vm-test-run-nixbot-gitlab> gitlab # [    2.553074] systemd[1]: Listening on udev Control Socket.605vm-test-run-nixbot-gitlab> gitlab # [    2.554971] systemd[1]: Listening on udev Kernel Socket.606vm-test-run-nixbot-gitlab> gitlab # [    2.556695] systemd[1]: Reached target Socket Units.607vm-test-run-nixbot-gitlab> gitlab # [    2.563253] systemd[1]: Starting Create List of Static Device Nodes...608vm-test-run-nixbot-gitlab> gitlab # [    2.569414] systemd[1]: Starting Load Kernel Module 9pnet_virtio...609vm-test-run-nixbot-gitlab> gitlab # [    2.589521] systemd[1]: Starting Load Kernel Module configfs...610vm-test-run-nixbot-gitlab> gitlab # [    2.595517] systemd[1]: Starting Journal Service...611vm-test-run-nixbot-gitlab> gitlab # [    2.599435] systemd[1]: Starting Load Kernel Modules...612vm-test-run-nixbot-gitlab> gitlab # [    2.602979] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki613vm-test-run-nixbot-gitlab> gitlab # [    2.611248] systemd[1]: Starting Coldplug All udev Devices...614vm-test-run-nixbot-gitlab> gitlab # [    2.643176] netfs: FS-Cache loaded615vm-test-run-nixbot-gitlab> gitlab # [    2.645643] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.616vm-test-run-nixbot-gitlab> gitlab # [    2.648992] systemd[1]: Finished Create List of Static Device Nodes.617vm-test-run-nixbot-gitlab> gitlab # [    2.649610] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev618vm-test-run-nixbot-gitlab> gitlab # [    2.650429] systemd-journald[152]: Collecting audit messages is disabled.619vm-test-run-nixbot-gitlab> gitlab # [    2.658467] systemd[1]: modprobe@configfs.service: Deactivated successfully.620vm-test-run-nixbot-gitlab> gitlab # [    2.658981] 9pnet: Installing 9P2000 support621vm-test-run-nixbot-gitlab> gitlab # [    2.662393] systemd[1]: Finished Load Kernel Module configfs.622vm-test-run-nixbot-gitlab> gitlab # [    2.670359] systemd[1]: Finished Load Kernel Modules.623vm-test-run-nixbot-gitlab> gitlab # [    2.672487] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.624vm-test-run-nixbot-gitlab> gitlab # [    2.676750] systemd[1]: Finished Load Kernel Module 9pnet_virtio.625vm-test-run-nixbot-gitlab> gitlab # [    2.682145] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config626vm-test-run-nixbot-gitlab> gitlab # [    2.687855] systemd[1]: Starting Apply Kernel Variables...627vm-test-run-nixbot-gitlab> gitlab # [    2.696282] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...628vm-test-run-nixbot-gitlab> gitlab # [    2.713606] systemd[1]: Finished Apply Kernel Variables.629vm-test-run-nixbot-gitlab> gitlab # [    2.458550] systemd-modules-load[154]: Using 4 probe threads630vm-test-run-nixbot-gitlab> gitlab # [    2.460905] systemd-modules-load[154]: Inserted module 'virtio_balloon'631vm-test-run-nixbot-gitlab> gitlab # [    2.723493] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.632vm-test-run-nixbot-gitlab> gitlab # [    2.462881] systemd-modules-load[154]: Inserted module 'virtio_gpu'633vm-test-run-nixbot-gitlab> gitlab # [    2.465699] systemd-modules-load[154]: Inserted module 'dm_mod'634vm-test-run-nixbot-gitlab> gitlab # [    2.728541] systemd[1]: Started Journal Service.635vm-test-run-nixbot-gitlab> gitlab # [    2.475492] systemd[1]: Starting Create Static Device Nodes in /dev...636vm-test-run-nixbot-gitlab> gitlab # [    2.495569] systemd[1]: Finished Create Static Device Nodes in /dev.637vm-test-run-nixbot-gitlab> gitlab # [    2.497569] systemd[1]: Reached target Preparation for Local File Systems.638vm-test-run-nixbot-gitlab> gitlab # [    2.499390] systemd[1]: Reached target Local File Systems.639vm-test-run-nixbot-gitlab> gitlab # [    2.500859] systemd[1]: Starting Create System Files and Directories...640vm-test-run-nixbot-gitlab> gitlab # [    2.502638] systemd[1]: Starting Rule-based Manager for Device Events and Files...641vm-test-run-nixbot-gitlab> gitlab # [    2.513088] systemd[1]: Finished Create System Files and Directories.642vm-test-run-nixbot-gitlab> gitlab # [    2.523257] systemd-udevd[192]: Using default interface naming scheme 'v260'.643vm-test-run-nixbot-gitlab> gitlab # [    2.540100] systemd[1]: Started Rule-based Manager for Device Events and Files.644vm-test-run-nixbot-gitlab> gitlab # [    2.544703] systemd[1]: Finished Coldplug All udev Devices.645vm-test-run-nixbot-gitlab> gitlab # [    2.546311] systemd[1]: Reached target System Initialization.646vm-test-run-nixbot-gitlab> gitlab # [    2.547839] systemd[1]: Reached target Basic System.647vm-test-run-nixbot-gitlab> gitlab # [    2.981137] virtio_blk virtio4: 4/0/0 default/read/poll queues648vm-test-run-nixbot-gitlab> gitlab # [    2.994671] uhci_hcd 0000:00:01.2: UHCI Host Controller649vm-test-run-nixbot-gitlab> gitlab # [    3.000275] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1650vm-test-run-nixbot-gitlab> gitlab # [    3.002132] uhci_hcd 0000:00:01.2: detected 2 ports651vm-test-run-nixbot-gitlab> gitlab # [    3.003717] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140652vm-test-run-nixbot-gitlab> gitlab # [    3.006466] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18653vm-test-run-nixbot-gitlab> gitlab # [    3.006704] SCSI subsystem initialized654vm-test-run-nixbot-gitlab> gitlab # [    3.007453] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1655vm-test-run-nixbot-gitlab> gitlab # [    3.007457] usb usb1: Product: UHCI Host Controller656vm-test-run-nixbot-gitlab> gitlab # [    3.007458] usb usb1: Manufacturer: Linux 6.18.34 uhci_hcd657vm-test-run-nixbot-gitlab> gitlab # [    3.014765] usb usb1: SerialNumber: 0000:00:01.2658vm-test-run-nixbot-gitlab> gitlab # [    3.016518] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)659vm-test-run-nixbot-gitlab> gitlab # [    3.036417] hub 1-0:1.0: USB hub found660vm-test-run-nixbot-gitlab> gitlab # [    3.037745] hub 1-0:1.0: 2 ports detected661vm-test-run-nixbot-gitlab> gitlab # [    3.048937] virtio_blk virtio5: 4/0/0 default/read/poll queues662vm-test-run-nixbot-gitlab> gitlab # [    3.052924] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12663vm-test-run-nixbot-gitlab> gitlab # [    3.057297] serio: i8042 KBD port at 0x60,0x64 irq 1664vm-test-run-nixbot-gitlab> gitlab # [    3.058725] serio: i8042 AUX port at 0x60,0x64 irq 12665vm-test-run-nixbot-gitlab> gitlab # [    3.059084] scsi host0: ata_piix666vm-test-run-nixbot-gitlab> gitlab # [    3.061917] virtio_blk virtio5: [vdb] 11449992 512-byte logical blocks (5.86 GB/5.46 GiB)667vm-test-run-nixbot-gitlab> gitlab # [    3.061969] scsi host1: ata_piix668vm-test-run-nixbot-gitlab> gitlab # [    3.065734] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0669vm-test-run-nixbot-gitlab> gitlab # [    3.067783] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0670vm-test-run-nixbot-gitlab> gitlab # [    2.830718] (udev-worker)[211]: Network interface NamePolicy= disabled on kernel command line.671vm-test-run-nixbot-gitlab> gitlab # [    2.834847] (udev-worker)[206]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.672vm-test-run-nixbot-gitlab> gitlab # [    2.838796] (udev-worker)[206]: Network interface NamePolicy= disabled on kernel command line.673vm-test-run-nixbot-gitlab> gitlab # [    2.849856] systemd[1]: Starting Virtual Console Setup...674vm-test-run-nixbot-gitlab> gitlab # [    2.863812] systemd[1]: Found device /dev/disk/by-label/nixos.675vm-test-run-nixbot-gitlab> gitlab # [    2.869103] systemd[1]: Found device /dev/disk/by-label/nix-store.676vm-test-run-nixbot-gitlab> gitlab # [    2.870909] systemd[1]: Reached target Initrd Root Device.677vm-test-run-nixbot-gitlab> gitlab # [    2.876344] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...678vm-test-run-nixbot-gitlab> gitlab # [    2.878515] systemd-vconsole-setup[233]: Configuration of first virtual console was skipped, ignoring remaining ones.679vm-test-run-nixbot-gitlab> gitlab # [    2.882910] systemd[1]: Finished Virtual Console Setup.680vm-test-run-nixbot-gitlab> gitlab # [    2.910521] systemd-fsck[241]: nixos: clean, 12/65536 files, 13019/262144 blocks681vm-test-run-nixbot-gitlab> gitlab # [    2.917255] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.682vm-test-run-nixbot-gitlab> gitlab # [    3.227400] ata2: found unknown device (class 0)683vm-test-run-nixbot-gitlab> gitlab # [    3.229959] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100684vm-test-run-nixbot-gitlab> gitlab # [    3.236326] scsi 1:0:0:0: CD-ROM            QEMU     QEMU DVD-ROM     2.5+ PQ: 0 ANSI: 5685vm-test-run-nixbot-gitlab> gitlab # [    3.267006] usb 1-1: new full-speed USB device number 2 using uhci_hcd686vm-test-run-nixbot-gitlab> gitlab # [    3.277941] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0687vm-test-run-nixbot-gitlab> gitlab # [    3.313610] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray688vm-test-run-nixbot-gitlab> gitlab # [    3.342976] cdrom: Uniform CD-ROM driver Revision: 3.20689vm-test-run-nixbot-gitlab> gitlab # [    3.436007] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00690vm-test-run-nixbot-gitlab> gitlab # [    3.440957] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10691vm-test-run-nixbot-gitlab> gitlab # [    3.445358] usb 1-1: Product: QEMU USB Tablet692vm-test-run-nixbot-gitlab> gitlab # [    3.448163] usb 1-1: Manufacturer: QEMU693vm-test-run-nixbot-gitlab> gitlab # [    3.450680] usb 1-1: SerialNumber: 28754-0000:00:01.2-1694vm-test-run-nixbot-gitlab> gitlab # [    3.494595] hid: raw HID events driver (C) Jiri Kosina695vm-test-run-nixbot-gitlab> gitlab # [    3.512118] usbcore: registered new interface driver usbhid696vm-test-run-nixbot-gitlab> gitlab # [    3.516072] usbhid: USB HID core driver697vm-test-run-nixbot-gitlab> gitlab # [    3.533950] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2698vm-test-run-nixbot-gitlab> gitlab # [    3.541555] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0699vm-test-run-nixbot-gitlab> gitlab # [    3.388215] systemd[1]: Mounting /sysroot...700vm-test-run-nixbot-gitlab> gitlab # [    3.831309] EXT4-fs (vda): mounted filesystem 1e24690e-db02-49d0-bfc9-d6ae9bcd003d r/w with ordered data mode. Quota mode: none.701vm-test-run-nixbot-gitlab> gitlab # [    3.574878] systemd[1]: Mounted /sysroot.702vm-test-run-nixbot-gitlab> gitlab # [    3.576534] systemd[1]: Reached target Initrd Root File System.703vm-test-run-nixbot-gitlab> gitlab # [    3.582057] systemd[1]: Mounting /sysroot/nix/.ro-store...704vm-test-run-nixbot-gitlab> gitlab # [    3.586846] systemd[1]: Mounting /sysroot/nix/.rw-store...705vm-test-run-nixbot-gitlab> gitlab # [    3.593511] systemd[1]: Mounting /sysroot/run...706vm-test-run-nixbot-gitlab> gitlab # [    3.602293] systemd[1]: Mounting /sysroot/tmp/shared...707vm-test-run-nixbot-gitlab> gitlab # [    3.606968] systemd[1]: Mounting /sysroot/tmp/xchg...708vm-test-run-nixbot-gitlab> gitlab # [    3.614350] systemd[1]: Starting Mountpoints Configured in the Real Root...709vm-test-run-nixbot-gitlab> gitlab # [    3.882921] erofs (device vdb): mounted with root inode @ nid 128.710vm-test-run-nixbot-gitlab> gitlab # [    3.623449] systemd[1]: Mounted /sysroot/nix/.rw-store.711vm-test-run-nixbot-gitlab> gitlab # [    3.625214] systemd[1]: Mounted /sysroot/nix/.ro-store.712vm-test-run-nixbot-gitlab> gitlab # [    3.626766] systemd[1]: Mounted /sysroot/run.[    3.889764] 9p: Installing v9fs 9p2000 file system support713vm-test-run-nixbot-gitlab> gitlab # 714vm-test-run-nixbot-gitlab> gitlab # [    3.634080] systemd[1]: Starting rw-sysroot-nix-store.service...715vm-test-run-nixbot-gitlab> gitlab # [    3.636105] systemd-sysroot-fstab-check[271]: /sysroot should be mounted in the initrd, will request daemon-reload.716vm-test-run-nixbot-gitlab> gitlab # [    3.638692] systemd[1]: Mounted /sysroot/tmp/shared.717vm-test-run-nixbot-gitlab> gitlab # [    3.639975] systemd[1]: Mounted /sysroot/tmp/xchg.718vm-test-run-nixbot-gitlab> gitlab # [    3.641310] systemd[1]: Reload requested from client PID 271 ('systemd-sysroot') (unit initrd-parse-etc.service)...719vm-test-run-nixbot-gitlab> gitlab # [    3.643940] systemd[1]: Reloading...720vm-test-run-nixbot-gitlab> gitlab # [    3.857433] systemd[1]: Reloading finished in 216 ms.721vm-test-run-nixbot-gitlab> gitlab # [    3.893786] systemd-sysroot-fstab-check[271]: Requesting initrd-fs.target/start/replace...722vm-test-run-nixbot-gitlab> gitlab # [    3.993958] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.723vm-test-run-nixbot-gitlab> gitlab # [    3.998979] systemd[1]: Finished rw-sysroot-nix-store.service.724vm-test-run-nixbot-gitlab> gitlab # [    4.002960] systemd-sysroot-fstab-check[271]: Requesting swap.target/start/replace...725vm-test-run-nixbot-gitlab> gitlab # [    4.007956] systemd[1]: Starting rw-sysroot-nix-store.service...726vm-test-run-nixbot-gitlab> gitlab # [    4.012358] systemd[1]: initrd-parse-etc.service: Deactivated successfully.727vm-test-run-nixbot-gitlab> gitlab # [    4.016706] systemd[1]: Finished Mountpoints Configured in the Real Root.728vm-test-run-nixbot-gitlab> gitlab # [    4.020634] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.729vm-test-run-nixbot-gitlab> gitlab # [    4.024719] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.730vm-test-run-nixbot-gitlab> gitlab # [    4.028405] systemd[1]: Finished rw-sysroot-nix-store.service.731vm-test-run-nixbot-gitlab> gitlab # [    4.389344] systemd[1]: Mounting /sysroot/nix/store...732vm-test-run-nixbot-gitlab> gitlab # [    4.460515] systemd[1]: Mounted /sysroot/nix/store.733vm-test-run-nixbot-gitlab> gitlab # [    4.464484] systemd[1]: Reached target Initrd File Systems.734vm-test-run-nixbot-gitlab> gitlab # [    4.472359] systemd[1]: Starting Find NixOS closure...735vm-test-run-nixbot-gitlab> gitlab # [    4.479267] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...736vm-test-run-nixbot-gitlab> gitlab # [    4.512847] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.737vm-test-run-nixbot-gitlab> gitlab # [    4.517650] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.738vm-test-run-nixbot-gitlab> gitlab # [    4.539107] systemd[1]: Finished Find NixOS closure.739vm-test-run-nixbot-gitlab> gitlab # [    4.544642] systemd[1]: Reached target Initrd Default Target.740vm-test-run-nixbot-gitlab> gitlab # [    4.552219] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...741vm-test-run-nixbot-gitlab> gitlab # [    4.593600] systemd[1]: Stopped target Initrd Default Target.742vm-test-run-nixbot-gitlab> gitlab # [    4.597329] systemd[1]: Stopped target Basic System.743vm-test-run-nixbot-gitlab> gitlab # [    4.600595] systemd[1]: Stopped target Initrd Root Device.744vm-test-run-nixbot-gitlab> gitlab # [    4.604127] systemd[1]: Stopped target Path Units.745vm-test-run-nixbot-gitlab> gitlab # [    4.607201] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.746vm-test-run-nixbot-gitlab> gitlab # [    4.611732] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.747vm-test-run-nixbot-gitlab> gitlab # [    4.616371] systemd[1]: Stopped target Slice Units.748vm-test-run-nixbot-gitlab> gitlab # [    4.619642] systemd[1]: Stopped target Socket Units.749vm-test-run-nixbot-gitlab> gitlab # [    4.622793] systemd[1]: Stopped target System Initialization.750vm-test-run-nixbot-gitlab> gitlab # [    4.626316] systemd[1]: Stopped target Swaps.751vm-test-run-nixbot-gitlab> gitlab # [    4.628946] systemd[1]: Stopped target Timer Units.752vm-test-run-nixbot-gitlab> gitlab # [    4.631770] systemd[1]: dbus.socket: Deactivated successfully.753vm-test-run-nixbot-gitlab> gitlab # [    4.634661] systemd[1]: Closed D-Bus System Message Bus Socket.754vm-test-run-nixbot-gitlab> gitlab # [    4.637464] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.755vm-test-run-nixbot-gitlab> gitlab # [    4.641109] systemd[1]: Stopped Find NixOS closure.756vm-test-run-nixbot-gitlab> gitlab # [    4.643645] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio757vm-test-run-nixbot-gitlab> gitlab # [    4.648500] systemd[1]: Starting rw-sysroot-nix-store.service...758vm-test-run-nixbot-gitlab> gitlab # [    4.651120] systemd[1]: systemd-sysctl.service: Deactivated successfully.759vm-test-run-nixbot-gitlab> gitlab # [    4.654153] systemd[1]: Stopped Apply Kernel Variables.760vm-test-run-nixbot-gitlab> gitlab # [    4.655577] systemd[1]: systemd-modules-load.service: Deactivated successfully.761vm-test-run-nixbot-gitlab> gitlab # [    4.657311] systemd[1]: Stopped Load Kernel Modules.762vm-test-run-nixbot-gitlab> gitlab # [    4.658588] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.763vm-test-run-nixbot-gitlab> gitlab # [    4.660486] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.764vm-test-run-nixbot-gitlab> gitlab # [    4.662373] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.765vm-test-run-nixbot-gitlab> gitlab # [    4.664166] systemd[1]: Stopped Create System Files and Directories.766vm-test-run-nixbot-gitlab> gitlab # [    4.668055] systemd[1]: Stopped target Local File Systems.767vm-test-run-nixbot-gitlab> gitlab # [    4.669481] systemd[1]: Stopped target Preparation for Local File Systems.768vm-test-run-nixbot-gitlab> gitlab # [    4.671155] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.769vm-test-run-nixbot-gitlab> gitlab # [    4.672896] systemd[1]: Stopped Coldplug All udev Devices.770vm-test-run-nixbot-gitlab> gitlab # [    4.676165] systemd[1]: Stopping Rule-based Manager for Device Events and Files...771vm-test-run-nixbot-gitlab> gitlab # [    4.678271] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.772vm-test-run-nixbot-gitlab> gitlab # [    4.680068] systemd[1]: Stopped Virtual Console Setup.773vm-test-run-nixbot-gitlab> gitlab # [    4.687290] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.774vm-test-run-nixbot-gitlab> gitlab # [    4.689924] systemd[1]: Finished rw-sysroot-nix-store.service.775vm-test-run-nixbot-gitlab> gitlab # [    4.695859] systemd[1]: systemd-udevd.service: Deactivated successfully.776vm-test-run-nixbot-gitlab> gitlab # [    4.700973] systemd[1]: Stopped Rule-based Manager for Device Events and Files.777vm-test-run-nixbot-gitlab> gitlab # [    4.703451] systemd[1]: systemd-udevd.service: Consumed 1.154s CPU time over 2.194s wall clock time, 25.2M memory peak.778vm-test-run-nixbot-gitlab> gitlab # [    4.706697] systemd[1]: initrd-cleanup.service: Deactivated successfully.779vm-test-run-nixbot-gitlab> gitlab # [    4.710271] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.780vm-test-run-nixbot-gitlab> gitlab # [    4.720214] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.781vm-test-run-nixbot-gitlab> gitlab # [    4.722476] systemd[1]: Closed udev Control Socket.782vm-test-run-nixbot-gitlab> gitlab # [    4.728404] systemd[1]: Starting Cleanup udev Database...783vm-test-run-nixbot-gitlab> gitlab # [    4.730343] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.784vm-test-run-nixbot-gitlab> gitlab # [    4.732683] systemd[1]: Stopped Create Static Device Nodes in /dev.785vm-test-run-nixbot-gitlab> gitlab # [    4.734736] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.786vm-test-run-nixbot-gitlab> gitlab # [    4.737326] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.787vm-test-run-nixbot-gitlab> gitlab # [    4.739564] systemd[1]: kmod-static-nodes.service: Deactivated successfully.788vm-test-run-nixbot-gitlab> gitlab # [    4.741673] systemd[1]: Stopped Create List of Static Device Nodes.789vm-test-run-nixbot-gitlab> gitlab # [    4.759261] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.790vm-test-run-nixbot-gitlab> gitlab # [    4.763283] systemd[1]: Finished Cleanup udev Database.791vm-test-run-nixbot-gitlab> gitlab # [    4.768561] systemd[1]: Reached target Switch Root.792vm-test-run-nixbot-gitlab> gitlab # [    4.771182] systemd[1]: Starting NixOS Activation...793vm-test-run-nixbot-gitlab> gitlab # [    4.898810] initrd-nixos-activation-start[571]: booting system configuration /nix/store/igb97j0wm8q7h3ykz6g8nlk205nm0n7q-nixos-system-gitlab-test794vm-test-run-nixbot-gitlab> gitlab # [    4.944365] initrd-nixos-activation-start[571]: running activation script...795vm-test-run-nixbot-gitlab> gitlab # [    5.175983] initrd-nixos-activation-start[594]: setting up /etc...796vm-test-run-nixbot-gitlab> gitlab # [    5.313829] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.797vm-test-run-nixbot-gitlab> gitlab # [    5.322822] systemd[1]: Finished NixOS Activation.798vm-test-run-nixbot-gitlab> gitlab # [    5.336612] systemd[1]: Starting Switch Root...799vm-test-run-nixbot-gitlab> gitlab # [    5.374604] systemd[1]: Switching root.800vm-test-run-nixbot-gitlab> gitlab # [    6.040841] systemd-journald[152]: Received SIGTERM from PID 1 (systemd).801vm-test-run-nixbot-gitlab> gitlab # [    6.165477] NET: Registered PF_VSOCK protocol family802vm-test-run-nixbot-gitlab> gitlab # [    6.544108] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)803vm-test-run-nixbot-gitlab> gitlab # [    6.560627] systemd[1]: Detected virtualization kvm.804vm-test-run-nixbot-gitlab> gitlab # [    6.563831] systemd[1]: Detected architecture x86-64.805vm-test-run-nixbot-gitlab> gitlab # [    6.567180] systemd[1]: Detected first boot.806vm-test-run-nixbot-gitlab> gitlab # [    6.571310] systemd[1]: Initializing machine ID from random generator.807vm-test-run-nixbot-gitlab> gitlab # [    6.728984] systemd[1]: bpf-restrict-fs: LSM BPF program attached808vm-test-run-nixbot-gitlab> gitlab # [    6.802189] systemd[1]: Applying preset policy.809vm-test-run-nixbot-gitlab> gitlab # [    6.977672] systemd[1]: Populated /etc with preset unit settings.810vm-test-run-nixbot-gitlab> gitlab # [    7.101482] systemd[1]: initrd-switch-root.service: Deactivated successfully.811vm-test-run-nixbot-gitlab> gitlab # [    7.103781] systemd[1]: Stopped initrd-switch-root.service.812vm-test-run-nixbot-gitlab> gitlab # [    7.107410] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.813vm-test-run-nixbot-gitlab> gitlab # [    7.110659] systemd[1]: Created slice Slice /system/getty.814vm-test-run-nixbot-gitlab> gitlab # [    7.113045] systemd[1]: Created slice GitLab DevOps Platform Slice.815vm-test-run-nixbot-gitlab> gitlab # [    7.115322] systemd[1]: Created slice User and Session Slice.816vm-test-run-nixbot-gitlab> gitlab # [    7.116949] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.817vm-test-run-nixbot-gitlab> gitlab # [    7.119035] systemd[1]: Started Forward Password Requests to Wall Directory Watch.818vm-test-run-nixbot-gitlab> gitlab # [    7.121007] systemd[1]: Expecting device /dev/hvc0...819vm-test-run-nixbot-gitlab> gitlab # [    7.122289] systemd[1]: Expecting device /dev/ttyS0...820vm-test-run-nixbot-gitlab> gitlab # [    7.123658] systemd[1]: Reached target Local Encrypted Volumes.821vm-test-run-nixbot-gitlab> gitlab # [    7.125223] systemd[1]: Stopped target initrd-fs.target.822vm-test-run-nixbot-gitlab> gitlab # [    7.126586] systemd[1]: Stopped target initrd-root-fs.target.823vm-test-run-nixbot-gitlab> gitlab # [    7.128071] systemd[1]: Stopped target initrd-switch-root.target.824vm-test-run-nixbot-gitlab> gitlab # [    7.129676] systemd[1]: Reached target Virtual Machines and Containers.825vm-test-run-nixbot-gitlab> gitlab # [    7.131396] systemd[1]: Reached target Path Units.826vm-test-run-nixbot-gitlab> gitlab # [    7.132714] systemd[1]: Reached target Remote File Systems.827vm-test-run-nixbot-gitlab> gitlab # [    7.134226] systemd[1]: Reached target Slice Units.828vm-test-run-nixbot-gitlab> gitlab # [    7.135572] systemd[1]: Reached target Swaps.829vm-test-run-nixbot-gitlab> gitlab # [    7.138019] systemd[1]: Listening on Process Core Dump Socket.830vm-test-run-nixbot-gitlab> gitlab # [    7.140160] systemd[1]: Listening on Credential Encryption/Decryption.831vm-test-run-nixbot-gitlab> gitlab # [    7.143690] systemd[1]: Starting Journal Log Access Socket...832vm-test-run-nixbot-gitlab> gitlab # [    7.145938] systemd[1]: Listening on Journal Audit Socket.833vm-test-run-nixbot-gitlab> gitlab # [    7.147635] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.834vm-test-run-nixbot-gitlab> gitlab # [    7.149556] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki835vm-test-run-nixbot-gitlab> gitlab # [    7.152042] systemd[1]: Listening on udev Control Socket.836vm-test-run-nixbot-gitlab> gitlab # [    7.155806] systemd[1]: Mounting Huge Pages File System...837vm-test-run-nixbot-gitlab> gitlab # [    7.158907] systemd[1]: Mounting POSIX Message Queue File System...838vm-test-run-nixbot-gitlab> gitlab # [    7.161992] systemd[1]: Mounting Kernel Debug File System...839vm-test-run-nixbot-gitlab> gitlab # [    7.166504] systemd[1]: Mounting Kernel Trace File System...840vm-test-run-nixbot-gitlab> gitlab # [    7.176076] systemd[1]: Starting Create List of Static Device Nodes...841vm-test-run-nixbot-gitlab> gitlab # [    7.178021] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio842vm-test-run-nixbot-gitlab> gitlab # [    7.185005] systemd[1]: Starting Load Kernel Module configfs...843vm-test-run-nixbot-gitlab> gitlab # [    7.186519] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm844vm-test-run-nixbot-gitlab> gitlab # [    7.191991] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore845vm-test-run-nixbot-gitlab> gitlab # [    7.195560] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse846vm-test-run-nixbot-gitlab> gitlab # [    7.205410] systemd[1]: Mounting FUSE Control File System...847vm-test-run-nixbot-gitlab> gitlab # [    7.209463] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67848vm-test-run-nixbot-gitlab> gitlab # [    7.244043] systemd[1]: Starting Journal Service...849vm-test-run-nixbot-gitlab> gitlab # [    7.246935] systemd[1]: Starting Load Kernel Modules...850vm-test-run-nixbot-gitlab> gitlab # [    7.250917] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...851vm-test-run-nixbot-gitlab> gitlab # [    7.256006] systemd[1]: Starting Remount Root and Kernel File Systems...852vm-test-run-nixbot-gitlab> gitlab # [    7.261069] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki853vm-test-run-nixbot-gitlab> gitlab # [    7.265992] systemd[1]: Starting Coldplug All udev Devices...854vm-test-run-nixbot-gitlab> gitlab # [    7.273727] systemd[1]: Listening on Journal Log Access Socket.855vm-test-run-nixbot-gitlab> gitlab # [    7.275631] systemd[1]: Mounted Huge Pages File System.856vm-test-run-nixbot-gitlab> gitlab # [    7.277339] systemd[1]: Mounted POSIX Message Queue File System.857vm-test-run-nixbot-gitlab> gitlab # [    7.279086] systemd[1]: Mounted Kernel Debug File System.858vm-test-run-nixbot-gitlab> gitlab # [    7.280692] systemd[1]: Mounted Kernel Trace File System.859vm-test-run-nixbot-gitlab> gitlab # [    7.283731] systemd-journald[790]: Collecting audit messages is enabled.860vm-test-run-nixbot-gitlab> gitlab # [    7.288135] loop: module loaded861vm-test-run-nixbot-gitlab> gitlab # [    7.288415] systemd[1]: Finished Create List of Static Device Nodes.862vm-test-run-nixbot-gitlab> gitlab # [    7.291984] EXT4-fs (vda): re-mounted 1e24690e-db02-49d0-bfc9-d6ae9bcd003d.863vm-test-run-nixbot-gitlab> gitlab # [    7.294144] systemd[1]: modprobe@configfs.service: Deactivated successfully.864vm-test-run-nixbot-gitlab> gitlab # [    7.297744] systemd[1]: Finished Load Kernel Module configfs.865vm-test-run-nixbot-gitlab> gitlab # [    7.300588] systemd[1]: Mounted FUSE Control File System.866vm-test-run-nixbot-gitlab> gitlab # [    7.303418] systemd[1]: Finished Remount Root and Kernel File Systems.867vm-test-run-nixbot-gitlab> gitlab # [    7.306633] systemd[1]: Finished Load Kernel Modules.868vm-test-run-nixbot-gitlab> gitlab # [    7.310119] systemd[1]: Mounting Kernel Configuration File System...869vm-test-run-nixbot-gitlab> gitlab # [    7.049716] systemd[1]: Queued start job for default target Multi-User System.870vm-test-run-nixbot-gitlab> gitlab # [    7.052743] systemd[1]: Unnecessary job was removed for /dev/vdb.871vm-test-run-nixbot-gitlab> gitlab # [    7.317016] systemd[1]: Starting Firewall...872vm-test-run-nixbot-gitlab> gitlab # [    7.054992] systemd[1]: systemd-journald.service: Deactivated[    7.318247] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore873vm-test-run-nixbot-gitlab> gitlab #  successfully.874vm-test-run-nixbot-gitlab> gitlab # [    7.059834] systemd-modules-load[791]: Using 3 probe threads875vm-test-run-nixbot-gitlab> gitlab # [    7.061843] systemd-modules-load[791]: Insert[    7.324981] systemd[1]: Starting Load/Save OS Random Seed...876vm-test-run-nixbot-gitlab> gitlab # ed module 'loop'877vm-test-run-nixbot-gitlab> gitlab # [    7.064875] systemd-modules-load[791]: Inserted module 'tls'878vm-test-run-nixbot-gitlab> gitlab # [    7.066939] systemd-oomd[792]: No swap; memory pressure usage will be degraded879vm-test-run-nixbot-gitlab> gitlab # [    7.336994] systemd[1]: Starting Apply Kernel Variables...880vm-test-run-nixbot-gitlab> gitlab # [    7.341024] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...881vm-test-run-nixbot-gitlab> gitlab # [    7.345066] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki882vm-test-run-nixbot-gitlab> gitlab # [    7.349075] systemd[1]: Started Journal Service.883vm-test-run-nixbot-gitlab> gitlab # [    7.088952] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.884vm-test-run-nixbot-gitlab> gitlab # [    7.092370] systemd[1]: Mounted Kernel Configuration File System.885vm-test-run-nixbot-gitlab> gitlab # [    7.094322] systemd[1]: Finished Load/Save OS Random Seed.886vm-test-run-nixbot-gitlab> gitlab # [    7.102094] systemd[1]: Reached target First Boot Complete.887vm-test-run-nixbot-gitlab> gitlab # [    7.109280] systemd[1]: Starting Flush Journal to Persistent Storage...888vm-test-run-nixbot-gitlab> gitlab # [    7.112394] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.889vm-test-run-nixbot-gitlab> gitlab # [    7.114950] systemd[1]: Finished Apply Kernel Variables.890vm-test-run-nixbot-gitlab> gitlab # [    7.125737] systemd[1]: Starting Create Static Device Nodes in /dev...891vm-test-run-nixbot-gitlab> gitlab # [    7.394048] systemd-journald[790]: Received client request to flush runtime journal.892vm-test-run-nixbot-gitlab> gitlab # [    7.188901] systemd[1]: Finished Create Static Device Nodes in /dev.893vm-test-run-nixbot-gitlab> gitlab # [    7.190956] systemd[1]: Reached target Preparation for Local File Systems.894vm-test-run-nixbot-gitlab> gitlab # [    7.192935] systemd[1]: Starting Rule-based Manager for Device Events and Files...895vm-test-run-nixbot-gitlab> gitlab # [    7.194939] systemd[1]: Finished Flush Journal to Persistent Storage.896vm-test-run-nixbot-gitlab> gitlab # [    7.206272] systemd-udevd[833]: Using default interface naming scheme 'v260'.897vm-test-run-nixbot-gitlab> gitlab # [    7.227059] systemd[1]: Finished Coldplug All udev Devices.898vm-test-run-nixbot-gitlab> gitlab # [    7.242907] systemd[1]: Started Rule-based Manager for Device Events and Files.899vm-test-run-nixbot-gitlab> gitlab # [    7.364991] systemd[1]: Finished Firewall.900vm-test-run-nixbot-gitlab> gitlab # [    7.369974] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse901vm-test-run-nixbot-gitlab> gitlab # [    7.393986] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.902vm-test-run-nixbot-gitlab> gitlab # [    7.400742] (udev-worker)[936]: Network interface NamePolicy= disabled on kernel command line.903vm-test-run-nixbot-gitlab> gitlab # [    7.426697] (udev-worker)[917]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.904vm-test-run-nixbot-gitlab> gitlab # [    7.430441] (udev-worker)[917]: Network interface NamePolicy= disabled on kernel command line.905vm-test-run-nixbot-gitlab> gitlab # [    7.449761] systemd[1]: Condition check resulted in Virtio network device being skipped.906vm-test-run-nixbot-gitlab> gitlab # [    7.454092] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore907vm-test-run-nixbot-gitlab> gitlab # [    7.457734] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67908vm-test-run-nixbot-gitlab> gitlab # [    7.462275] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore909vm-test-run-nixbot-gitlab> gitlab # [    7.465361] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki910vm-test-run-nixbot-gitlab> gitlab # [    7.467855] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki911vm-test-run-nixbot-gitlab> gitlab # [    7.486946] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped.912vm-test-run-nixbot-gitlab> gitlab # [    7.756373] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3913vm-test-run-nixbot-gitlab> gitlab # [    7.762286] ACPI: button: Power Button [PWRF]914vm-test-run-nixbot-gitlab> gitlab # [    7.787672] rtc_cmos 00:05: RTC can wake from S4915vm-test-run-nixbot-gitlab> gitlab # [    7.808200] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0916vm-test-run-nixbot-gitlab> gitlab # [    7.810265] parport_pc 00:03: reported by Plug and Play ACPI917vm-test-run-nixbot-gitlab> gitlab # [    7.810283] rtc_cmos 00:05: registered as rtc0918vm-test-run-nixbot-gitlab> gitlab # [    7.810532] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD919vm-test-run-nixbot-gitlab> gitlab # [    7.810769] parport0: PC-style at 0x378920vm-test-run-nixbot-gitlab> gitlab # [    7.811132] rtc_cmos 00:05: setting system clock to 2026-07-04T12:35:48 UTC (1783168548)921vm-test-run-nixbot-gitlab> gitlab # [    7.811390] , irq 7922vm-test-run-nixbot-gitlab> gitlab # [    7.811807] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs923vm-test-run-nixbot-gitlab> gitlab # [    7.811901] systemd-journald[790]: Time jumped backwards, rotating.924vm-test-run-nixbot-gitlab> gitlab # [    7.812198]  [925vm-test-run-nixbot-gitlab> gitlab # [    7.815370] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console926vm-test-run-nixbot-gitlab> gitlab # [    7.815674] PCSPP(,...)]927vm-test-run-nixbot-gitlab> gitlab # [    7.840603] Console: switching to colour dummy device 80x25928vm-test-run-nixbot-gitlab> gitlab # [    7.842533] [drm] Found bochs VGA, ID 0xb0c5.929vm-test-run-nixbot-gitlab> gitlab # [    7.842536] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000.930vm-test-run-nixbot-gitlab> gitlab # [    7.844300] Floppy drive(s): fd0 is 2.88M AMI BIOS931vm-test-run-nixbot-gitlab> gitlab # [    7.859691] FDC 0 is a S82078B932vm-test-run-nixbot-gitlab> gitlab # [    7.863008] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic933vm-test-run-nixbot-gitlab> gitlab # [    7.864614] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0934vm-test-run-nixbot-gitlab> gitlab # [    7.868211] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5935vm-test-run-nixbot-gitlab> gitlab # [    7.871939] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input6936vm-test-run-nixbot-gitlab> gitlab # [    7.873828] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4937vm-test-run-nixbot-gitlab> gitlab # [    7.875934] fbcon: bochs-drmdrmfb (fb0) is primary device938vm-test-run-nixbot-gitlab> gitlab # [    7.892514] Console: switching to colour frame buffer device 160x50939vm-test-run-nixbot-gitlab> gitlab # [    7.896229] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device940vm-test-run-nixbot-gitlab> gitlab # [    7.938191] mousedev: PS/2 mouse device common for all mice941vm-test-run-nixbot-gitlab> gitlab # [    7.717475] systemd[1]: Starting Virtual Console Setup...942vm-test-run-nixbot-gitlab> gitlab # [    7.744754] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.943vm-test-run-nixbot-gitlab> gitlab # [    7.747693] systemd[1]: Stopped Virtual Console Setup.944vm-test-run-nixbot-gitlab> gitlab # [    8.020280] kvm_amd: TSC scaling supported945vm-test-run-nixbot-gitlab> gitlab # [    8.021296] kvm_amd: Nested Virtualization enabled946vm-test-run-nixbot-gitlab> gitlab # [    8.022449] kvm_amd: Nested Paging enabled947vm-test-run-nixbot-gitlab> gitlab # [    8.023401] kvm_amd: LBR virtualization supported948vm-test-run-nixbot-gitlab> gitlab # [    7.760695] systemd[1]: Starting Virtual Console Setup...949vm-test-run-nixbot-gitlab> gitlab # [    8.026252] kvm_amd: Virtual VMLOAD VMSAVE supported950vm-test-run-nixbot-gitlab> gitlab # [    8.027356] kvm_amd: Virtual GIF supported951vm-test-run-nixbot-gitlab> gitlab # [    8.028943] kvm_amd: Virtual NMI enabled952vm-test-run-nixbot-gitlab> gitlab # [    7.778974] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.953vm-test-run-nixbot-gitlab> gitlab # [    7.783449] systemd[1]: Stopped Virtual Console Setup.954vm-test-run-nixbot-gitlab> gitlab # [    7.790472] systemd[1]: Starting Virtual Console Setup...955vm-test-run-nixbot-gitlab> gitlab # [    8.059559] ppdev: user-space parallel port driver956vm-test-run-nixbot-gitlab> gitlab # [    8.078619] EDAC MC: Ver: 3.0.0957vm-test-run-nixbot-gitlab> gitlab # [    7.840601] systemd[1]: Mounting /run/wrappers...958vm-test-run-nixbot-gitlab> gitlab # [    7.856860] systemd-vconsole-setup[967]: Configuration of first virtual console was skipped, ignoring remaining ones.959vm-test-run-nixbot-gitlab> gitlab # [    7.861141] systemd[1]: Finished Virtual Console Setup.960vm-test-run-nixbot-gitlab> gitlab # [    7.866428] systemd[1]: Mounted /run/wrappers.961vm-test-run-nixbot-gitlab> gitlab # [    7.867691] systemd[1]: Reached target Local File Systems.962vm-test-run-nixbot-gitlab> gitlab # [    7.869150] systemd[1]: Listening on Boot Loader Control Service Socket.963vm-test-run-nixbot-gitlab> gitlab # [    7.870872] systemd[1]: Starting register-nix-paths.service...964vm-test-run-nixbot-gitlab> gitlab # [    7.872402] systemd[1]: Starting Create SUID/SGID Wrappers...965vm-test-run-nixbot-gitlab> gitlab # [    7.874043] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.966vm-test-run-nixbot-gitlab> gitlab # [    7.876315] systemd[1]: Starting Save Transient machine-id to Disk...967vm-test-run-nixbot-gitlab> gitlab # [    7.885917] systemd[1]: Starting Create System Files and Directories...968vm-test-run-nixbot-gitlab> gitlab # [    7.910122] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.969vm-test-run-nixbot-gitlab> gitlab # [    7.912448] systemd[1]: Finished Save Transient machine-id to Disk.970vm-test-run-nixbot-gitlab> gitlab # [    7.923167] systemd[1]: Finished Create System Files and Directories.971vm-test-run-nixbot-gitlab> gitlab # [    7.927386] systemd[1]: Starting Rebuild Journal Catalog...972vm-test-run-nixbot-gitlab> gitlab # [    7.929464] systemd[1]: Starting Record System Boot/Shutdown in UTMP...973vm-test-run-nixbot-gitlab> gitlab # [    7.946420] systemd[1]: Finished Record System Boot/Shutdown in UTMP.974vm-test-run-nixbot-gitlab> gitlab # [    7.962954] systemd[1]: Finished Rebuild Journal Catalog.975vm-test-run-nixbot-gitlab> gitlab # [    7.966947] systemd[1]: Starting Update is Completed...976vm-test-run-nixbot-gitlab> gitlab # [    7.982108] systemd[1]: Finished Update is Completed.977vm-test-run-nixbot-gitlab> gitlab # [    8.033347] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.978vm-test-run-nixbot-gitlab> gitlab # [    8.035644] systemd[1]: Finished Create SUID/SGID Wrappers.979vm-test-run-nixbot-gitlab> gitlab # [    8.037226] systemd[1]: suid-sgid-wrappers.service: Consumed 128ms CPU time over 160ms wall clock time, 4.4M memory peak, 1.1M read from disk.980vm-test-run-nixbot-gitlab> gitlab # [    8.285218] systemd[1]: Finished register-nix-paths.service.981vm-test-run-nixbot-gitlab> gitlab # [    8.290819] systemd[1]: Reached target System Initialization.982vm-test-run-nixbot-gitlab> gitlab # [    8.295113] systemd[1]: Started Discard unused filesystem blocks once a week.983vm-test-run-nixbot-gitlab> gitlab # [    8.299810] systemd[1]: Started logrotate.timer.984vm-test-run-nixbot-gitlab> gitlab # [    8.303351] systemd[1]: Started Daily Cleanup of Temporary Directories.985vm-test-run-nixbot-gitlab> gitlab # [    8.307326] systemd[1]: Reached target Timer Units.986vm-test-run-nixbot-gitlab> gitlab # [    8.310517] systemd[1]: Listening on D-Bus System Message Bus Socket.987vm-test-run-nixbot-gitlab> gitlab # [    8.314460] systemd[1]: Listening on Nix Daemon Socket.988vm-test-run-nixbot-gitlab> gitlab # [    8.320543] systemd[1]: Starting nixbot CI service socket...989vm-test-run-nixbot-gitlab> gitlab # [    8.324471] systemd[1]: Listening on Hostname Service Socket.990vm-test-run-nixbot-gitlab> gitlab # [    8.328203] systemd[1]: Listening on nixbot CI service socket.991vm-test-run-nixbot-gitlab> gitlab # [    8.331933] systemd[1]: Reached target Socket Units.992vm-test-run-nixbot-gitlab> gitlab # [    8.335251] systemd[1]: Starting D-Bus System Message Bus...993vm-test-run-nixbot-gitlab> gitlab # [    8.366683] dbus-broker-launch[1057]: Looking up NSS user entry for 'systemd-timesync'...994vm-test-run-nixbot-gitlab> gitlab # [    8.372509] dbus-broker-launch[1057]: NSS returned no entry for 'systemd-timesync'995vm-test-run-nixbot-gitlab> gitlab # [    8.376944] dbus-broker-launch[1057]: Invalid user-name in /nix/store/89zj9nmrajvcy9c3c08wv0ir9ww7lgmq-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"996vm-test-run-nixbot-gitlab> gitlab # [    8.386284] systemd[1]: Started D-Bus System Message Bus.997vm-test-run-nixbot-gitlab> gitlab # [    8.402267] dbus-broker-launch[1057]: Ready998vm-test-run-nixbot-gitlab> gitlab # [    8.406689] systemd[1]: Reached target Basic System.999vm-test-run-nixbot-gitlab> gitlab # [    8.410761] systemd[1]: Started backdoor.service.1000vm-test-run-nixbot-gitlab> gitlab # [    8.416947] systemd[1]: Starting gitlab-config.service...1001vm-test-run-nixbot-gitlab> gitlab # [    8.421190] systemd[1]: Starting Import lastlog data into lastlog2 database...1002vm-test-run-nixbot-gitlab> gitlab # [    8.429242] systemd[1]: Starting Logrotate configuration check...1003vm-test-run-nixbot-gitlab> gitlab # [    8.438189] systemd[1]: Starting Name Service Cache Daemon (nsncd)...1004vm-test-run-nixbot-gitlab> gitlab # [    8.443337] systemd[1]: Starting Post-Boot Actions...1005vm-test-run-nixbot-gitlab> gitlab # connecting to host...1006vm-test-run-nixbot-gitlab> gitlab # [    8.449934] systemd[1]: Started Reset console on configuration changes.1007vm-test-run-nixbot-gitlab> gitlab # [    8.452189] logrotate[1062]: warning: logrotate in debug mode does nothing except printing debug messages!  Consider using verbose mode (-v) instead if this is not what you want.1008vm-test-run-nixbot-gitlab> gitlab # [    8.453574] logrotate[1062]: reading config file /nix/store/sn4bd2nqi3yg4r51djm9i2rmfgbwvmhb-logrotate.conf1009vm-test-run-nixbot-gitlab> gitlab # [    8.455462] logrotate[1062]: note: 'monthly' overrides previously specified 'weekly'1010vm-test-run-nixbot-gitlab> gitlab # [    8.456908] logrotate[1062]: Reading state from file: /var/lib/logrotate.status1011vm-test-run-nixbot-gitlab> gitlab # [    8.457728] logrotate[1062]: state file /var/lib/logrotate.status does not exist1012vm-test-run-nixbot-gitlab> gitlab # [    8.459244] logrotate[1062]: Allocating hash table for state file, size 64 entries1013vm-test-run-nixbot-gitlab> gitlab # [    8.460398] logrotate[1062]: Handling 4 logs1014vm-test-run-nixbot-gitlab> gitlab # [    8.462403] logrotate[1062]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed1015vm-test-run-nixbot-gitlab> gitlab # [    8.463240] logrotate[1062]: considering log /var/log/btmp1016vm-test-run-nixbot-gitlab> gitlab # [    8.465241] logrotate[1062]: Creating new state1017vm-test-run-nixbot-gitlab> gitlab # [    8.466164] logrotate[1062]:   Now: 2026-07-04 12:351018vm-test-run-nixbot-gitlab> gitlab # [    8.467062] logrotate[1062]:   Last rotated at 2026-07-04 12:001019vm-test-run-nixbot-gitlab> gitlab # [    8.467927] logrotate[1062]:   log does not need rotating (log has already been rotated)1020vm-test-run-nixbot-gitlab> gitlab # [    8.469653] logrotate[1062]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed1021vm-test-run-nixbot-gitlab> gitlab # [    8.471278] logrotate[1062]: considering log /var/log/wtmp1022vm-test-run-nixbot-gitlab> gitlab # [    8.472193] logrotate[1062]: Creating new state1023vm-test-run-nixbot-gitlab> gitlab # [    8.472901] logrotate[1062]:   Now: 2026-07-04 12:351024vm-test-run-nixbot-gitlab> gitlab # [    8.474162] logrotate[1062]:   Last rotated at 2026-07-04 12:001025vm-test-run-nixbot-gitlab> gitlab # [    8.475372] logrotate[1062]:   log does not need rotating (log has already been rotated)1026vm-test-run-nixbot-gitlab> gitlab # [    8.476389] logrotate[1062]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed1027vm-test-run-nixbot-gitlab> gitlab # [    8.478415] logrotate[1062]: switching euid from 0 to 165 and egid from 0 to 165 (pid 1062)1028vm-test-run-nixbot-gitlab> gitlab # [    8.479293] logrotate[1062]: considering log /var/gitlab/state/log/*.log1029vm-test-run-nixbot-gitlab> gitlab # [    8.480163] logrotate[1062]:   log /var/gitlab/state/log/*.log does not exist -- skipping1030vm-test-run-nixbot-gitlab> gitlab # [    8.482426] logrotate[1062]: Creating new state1031vm-test-run-nixbot-gitlab> gitlab # [    8.483342] logrotate[1062]: switching euid from 165 to 0 and egid from 165 to 0 (pid 1062)1032vm-test-run-nixbot-gitlab> gitlab # [    8.484330] logrotate[1062]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed1033vm-test-run-nixbot-gitlab> gitlab # [    8.485652] logrotate[1062]: switching euid from 0 to 60 and egid from 0 to 60 (pid 1062)1034vm-test-run-nixbot-gitlab> gitlab # [    8.486903] logrotate[1062]: considering log /var/log/nginx/*.log1035vm-test-run-nixbot-gitlab> gitlab: Guest shell says: b'Spawning backdoor root shell...\n'1036vm-test-run-nixbot-gitlab> gitlab: connected to guest root shell1037vm-test-run-nixbot-gitlab> gitlab: (connecting took 9.37 seconds)1038vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for the VM to finish booting, in 9.37 seconds)1039vm-test-run-nixbot-gitlab> gitlab # [    8.500380] logrotate[1062]:   log /var/log/nginx/*.log does not exist -- skipping1040vm-test-run-nixbot-gitlab> gitlab # [    8.501794] logrotate[1062]: Creating new state1041vm-test-run-nixbot-gitlab> gitlab # [    8.503198] logrotate[1062]: switching euid from 60 to 0 and egid from 60 to 0 (pid 1062)1042vm-test-run-nixbot-gitlab> gitlab # [    8.504342] systemd[1]: Starting resolvconf update...1043vm-test-run-nixbot-gitlab> gitlab # [    8.505558] systemd[1]: Finished Logrotate configuration check.1044vm-test-run-nixbot-gitlab> gitlab # [    8.507168] systemd[1]: Finished Post-Boot Actions.1045vm-test-run-nixbot-gitlab> gitlab # [    8.525125] systemd[1]: Finished Import lastlog data into lastlog2 database.1046vm-test-run-nixbot-gitlab> gitlab # [    8.539400] nsncd[1064]: Jul 04 12:35:49.490 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"1047vm-test-run-nixbot-gitlab> gitlab # [    8.543267] systemd[1]: Started Name Service Cache Daemon (nsncd).1048vm-test-run-nixbot-gitlab> gitlab # [    8.545084] systemd[1]: Reached target Host and Network Name Lookups.1049vm-test-run-nixbot-gitlab> gitlab # [    8.546890] systemd[1]: Reached target User and Group Name Lookups.1050vm-test-run-nixbot-gitlab> gitlab # [    8.548656] systemd[1]: Starting User Login Management...1051vm-test-run-nixbot-gitlab> gitlab # [    8.563477] systemd[1]: Stopped target Host and Network Name Lookups.1052vm-test-run-nixbot-gitlab> gitlab # [    8.565338] systemd[1]: Stopping Host and Network Name Lookups...1053vm-test-run-nixbot-gitlab> gitlab # [    8.567123] systemd[1]: Stopped target User and Group Name Lookups.1054vm-test-run-nixbot-gitlab> gitlab # [    8.569055] systemd[1]: Stopping User and Group Name Lookups...1055vm-test-run-nixbot-gitlab> gitlab # [    8.570861] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...1056vm-test-run-nixbot-gitlab> gitlab # [    8.572632] systemd[1]: nscd.service: Deactivated successfully.1057vm-test-run-nixbot-gitlab> gitlab # [    8.574370] systemd[1]: Stopped Name Service Cache Daemon (nsncd).1058vm-test-run-nixbot-gitlab> gitlab # [    8.576070] systemd[1]: nscd.service: Consumed 62ms CPU time over 132ms wall clock time, 8.2M memory peak, 5.6M read from disk.1059vm-test-run-nixbot-gitlab> gitlab # [    8.579235] systemd[1]: Starting Name Service Cache Daemon (nsncd)...1060vm-test-run-nixbot-gitlab> gitlab # [    8.600791] systemd[1]: Finished resolvconf update.1061vm-test-run-nixbot-gitlab> gitlab # [    8.602157] systemd[1]: Reached target Preparation for Network.1062vm-test-run-nixbot-gitlab> gitlab # [    8.605737] nsncd[1151]: Jul 04 12:35:49.557 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"1063vm-test-run-nixbot-gitlab> gitlab # [    8.607661] systemd-logind[1131]: New seat seat0.1064vm-test-run-nixbot-gitlab> gitlab # [    8.609216] systemd[1]: Starting DHCP Client...1065vm-test-run-nixbot-gitlab> gitlab # [    8.613252] systemd[1]: Starting Address configuration of eth1...1066vm-test-run-nixbot-gitlab> gitlab # [    8.615646] systemd-logind[1131]: Watching system buttons on /dev/input/event2 (Power Button)1067vm-test-run-nixbot-gitlab> gitlab # [    8.617213] systemd-logind[1131]: Watching system buttons on /dev/input/event4 (QEMU Virtio Keyboard)1068vm-test-run-nixbot-gitlab> gitlab # [    8.618541] systemd-logind[1131]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard)1069vm-test-run-nixbot-gitlab> gitlab # [    8.619426] systemd[1]: Starting Extra networking commands....1070vm-test-run-nixbot-gitlab> gitlab # [    8.620834] systemd[1]: Started Name Service Cache Daemon (nsncd).1071vm-test-run-nixbot-gitlab> gitlab # [    8.623400] systemd[1]: Reached target Host and Network Name Lookups.1072vm-test-run-nixbot-gitlab> gitlab # [    8.625080] systemd[1]: Reached target User and Group Name Lookups.1073vm-test-run-nixbot-gitlab> gitlab # [    8.628224] systemd[1]: Started User Login Management.1074vm-test-run-nixbot-gitlab> gitlab # [    8.635983] systemd[1]: Starting linger-users.service...1075vm-test-run-nixbot-gitlab> gitlab # [    8.652637] systemd[1]: linger-users.service: Deactivated successfully.1076vm-test-run-nixbot-gitlab> gitlab # [    8.654929] systemd[1]: Finished linger-users.service.1077vm-test-run-nixbot-gitlab> gitlab # [    8.660958] network-addresses-eth1-start[1172]: adding address 192.168.1.1/24... done1078vm-test-run-nixbot-gitlab> gitlab # [    8.663915] network-addresses-eth1-start[1172]: adding address 2001:db8:1::1/64... done1079vm-test-run-nixbot-gitlab> gitlab # [    8.674695] systemd[1]: Finished Address configuration of eth1.1080vm-test-run-nixbot-gitlab> gitlab # [    8.709090] systemd[1]: Finished Extra networking commands..1081vm-test-run-nixbot-gitlab> gitlab # [    8.710853] systemd[1]: Reached target Network.1082vm-test-run-nixbot-gitlab> gitlab # [    8.712862] systemd[1]: Starting gitlab-workhorse.service...1083vm-test-run-nixbot-gitlab> gitlab # [    8.714838] dhcpcd[1183]: dhcpcd-10.3.2 starting1084vm-test-run-nixbot-gitlab> gitlab # [    8.716438] systemd[1]: Starting Nginx Web Server...1085vm-test-run-nixbot-gitlab> gitlab # [    8.720342] dhcpcd[1234]: dev: loaded udev1086vm-test-run-nixbot-gitlab> gitlab # [    8.721730] systemd[1]: Starting PostgreSQL Server...1087vm-test-run-nixbot-gitlab> gitlab # [    8.725840] systemd[1]: Starting Redis Server - redis-gitlab...1088vm-test-run-nixbot-gitlab> gitlab # [    8.728624] systemd[1]: Starting Permit User Sessions...1089vm-test-run-nixbot-gitlab> gitlab # [    8.994545] 8021q: 802.1Q VLAN Support v1.81090vm-test-run-nixbot-gitlab> gitlab # [    8.997923] 8021q: adding VLAN 0 to HW filter on device eth11091vm-test-run-nixbot-gitlab> gitlab # [    8.751912] systemd[1]: Finished Permit User Sessions.1092vm-test-run-nixbot-gitlab> gitlab # [    8.754693] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.1093vm-test-run-nixbot-gitlab> gitlab # [    8.758561] systemd[1]: Started Getty on tty1.1094vm-test-run-nixbot-gitlab> gitlab # [    8.762117] systemd[1]: Reached target Login Prompts.1095vm-test-run-nixbot-gitlab> gitlab # [    9.111808] cfg80211: Loading compiled-in X.509 certificates for regulatory database1096vm-test-run-nixbot-gitlab> gitlab # [    9.119377] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'1097vm-test-run-nixbot-gitlab> gitlab # [    9.121570] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'1098vm-test-run-nixbot-gitlab> gitlab # [    9.124532] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -21099vm-test-run-nixbot-gitlab> gitlab # [    9.126103] cfg80211: failed to load regulatory.db1100vm-test-run-nixbot-gitlab> gitlab # [    8.877671] nginx-pre-start[1259]: nginx: the configuration file /nix/store/nlpb2pnbsk0921apkv5yml2qxbm30d7w-nginx.conf syntax is ok1101vm-test-run-nixbot-gitlab> gitlab # [    8.878577] nginx-pre-start[1259]: nginx: configuration file /nix/store/nlpb2pnbsk0921apkv5yml2qxbm30d7w-nginx.conf test is successful1102vm-test-run-nixbot-gitlab> gitlab # [    8.893529] postgresql-pre-start[1260]: The files belonging to this database system will be owned by user "postgres".1103vm-test-run-nixbot-gitlab> gitlab # [    8.895143] postgresql-pre-start[1260]: This user must also own the server process.1104vm-test-run-nixbot-gitlab> gitlab # [    8.896666] redis-server[1256]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.1105vm-test-run-nixbot-gitlab> gitlab # [    8.898962] systemd[1]: Started Nginx Web Server.1106vm-test-run-nixbot-gitlab> gitlab # [    8.900651] postgresql-pre-start[1260]: The database cluster will be initialized with locale "en_US.UTF-8".1107vm-test-run-nixbot-gitlab> gitlab # [    8.901964] postgresql-pre-start[1260]: The default database encoding has accordingly been set to "UTF8".1108vm-test-run-nixbot-gitlab> gitlab # [    8.90[    9.170166] 8021q: adding VLAN 0 to HW filter on device eth01109vm-test-run-nixbot-gitlab> gitlab # 3682] postgresql-pre-start[1260]: The default text search configuration will be set to "english".1110vm-test-run-nixbot-gitlab> gitlab # [    8.904849] postgresql-pre-start[1260]: Data page checksums are disabled.1111vm-test-run-nixbot-gitlab> gitlab # [    8.908350] postgresql-pre-start[1260]: fixing permissions on existing directory /var/lib/postgresql/17 ... ok1112vm-test-run-nixbot-gitlab> gitlab # [    8.910833] postgresql-pre-start[1260]: creating subdirectories ... ok1113vm-test-run-nixbot-gitlab> gitlab # [    8.911807] postgresql-pre-start[1260]: selecting dynamic shared memory implementation ... posix1114vm-test-run-nixbot-gitlab> gitlab # [    8.913364] redis-server[1256]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo1115vm-test-run-nixbot-gitlab> gitlab # [    8.914254] systemd[1]: Started Redis Server - redis-gitlab.1116vm-test-run-nixbot-gitlab> gitlab # [    8.915218] redis-server[1256]: Redis version=8.8.0, bits=64, commit=00000000, modified=0, pid=1256, just started1117vm-test-run-nixbot-gitlab> gitlab # [    8.915801] redis-server[1256]: Configuration loaded1118vm-test-run-nixbot-gitlab> gitlab # [    8.916543] redis[1256]: monotonic clock: POSIX clock_gettime1119vm-test-run-nixbot-gitlab> gitlab # [    8.918166] redis[1256]: Running mode=standalone, port=0.1120vm-test-run-nixbot-gitlab> gitlab # [    8.918527] redis[1256]: Server initialized1121vm-test-run-nixbot-gitlab> gitlab # [    8.919409] redis[1256]: Ready to accept connections unix1122vm-test-run-nixbot-gitlab> gitlab # [    8.920095] redis[1256]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.1123vm-test-run-nixbot-gitlab> gitlab # [    8.920916] dhcpcd[1234]: eth0: waiting for carrier1124vm-test-run-nixbot-gitlab> gitlab # [    8.922268] dhcpcd[1234]: eth0: carrier acquired1125vm-test-run-nixbot-gitlab> gitlab # [    8.923161] dhcpcd[1234]: DUID 00:01:00:01:31:db:ba:a5:52:54:00:12:34:561126vm-test-run-nixbot-gitlab> gitlab # [    8.923731] dhcpcd[1234]: eth0: IAID 00:12:34:561127vm-test-run-nixbot-gitlab> gitlab # [    8.925180] dhcpcd[1234]: eth0: adding address fe80::5054:ff:fe12:34561128vm-test-run-nixbot-gitlab> gitlab # [    8.946228] postgresql-pre-start[1260]: selecting default "max_connections" ... 1001129vm-test-run-nixbot-gitlab> gitlab # [    8.992612] postgresql-pre-start[1260]: selecting default "shared_buffers" ... 128MB1130vm-test-run-nixbot-gitlab> gitlab # [    9.208622] systemd[1]: Started gitlab-workhorse.service.1131vm-test-run-nixbot-gitlab> gitlab # [    9.374161] postgresql-pre-start[1260]: selecting default time zone ... UTC1132vm-test-run-nixbot-gitlab> gitlab # [    9.376453] postgresql-pre-start[1260]: creating configuration files ... ok1133vm-test-run-nixbot-gitlab> gitlab # [    9.509167] gitlab-workhorse[1293]: time="2026-07-04T12:35:50Z" level=info msg=Starting build_time=19700101.000000 version=18.11.41134vm-test-run-nixbot-gitlab> gitlab # [    9.510379] gitlab-workhorse[1293]: time="2026-07-04T12:35:50Z" level=info msg="Using redis/go-redis"1135vm-test-run-nixbot-gitlab> gitlab # [    9.513183] gitlab-workhorse[1293]: time="2026-07-04T12:35:50Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix1136vm-test-run-nixbot-gitlab> gitlab # [    9.553858] postgresql-pre-start[1260]: running bootstrap script ... ok1137vm-test-run-nixbot-gitlab> gitlab # [   10.022783] postgresql-pre-start[1260]: performing post-bootstrap initialization ... ok1138vm-test-run-nixbot-gitlab> gitlab # [   10.409922] postgresql-pre-start[1260]: syncing data to disk ... ok1139vm-test-run-nixbot-gitlab> gitlab # [   10.412271] postgresql-pre-start[1260]: initdb: warning: enabling "trust" authentication for local connections1140vm-test-run-nixbot-gitlab> gitlab # [   10.413875] postgresql-pre-start[1260]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.1141vm-test-run-nixbot-gitlab> gitlab # [   10.415453] postgresql-pre-start[1260]: Success. You can now start the database server using:1142vm-test-run-nixbot-gitlab> gitlab # [   10.416911] postgresql-pre-start[1260]:     pg_ctl -D /var/lib/postgresql/17 -l logfile start1143vm-test-run-nixbot-gitlab> gitlab # [   10.526216] postgres[1319]: [1319] LOG:  starting PostgreSQL 17.10 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit1144vm-test-run-nixbot-gitlab> gitlab # [   10.532982] postgres[1319]: [1319] LOG:  listening on IPv6 address "::1", port 54321145vm-test-run-nixbot-gitlab> gitlab # [   10.534976] postgres[1319]: [1319] LOG:  listening on IPv4 address "127.0.0.1", port 54321146vm-test-run-nixbot-gitlab> gitlab # [   10.545574] postgres[1319]: [1319] LOG:  listening on Unix socket "/run/postgresql/.s.PGSQL.5432"1147vm-test-run-nixbot-gitlab> gitlab # [   10.561968] postgres[1325]: [1325] LOG:  database system was shut down at 2026-07-04 12:35:50 GMT1148vm-test-run-nixbot-gitlab> gitlab # [   10.575415] postgres[1319]: [1319] LOG:  database system is ready to accept connections1149vm-test-run-nixbot-gitlab> gitlab # [   10.583716] systemd[1]: Started PostgreSQL Server.1150vm-test-run-nixbot-gitlab> gitlab # [   10.587787] systemd[1]: Starting PostgreSQL Setup Scripts...1151vm-test-run-nixbot-gitlab> gitlab # [   10.698819] dhcpcd[1234]: eth0: soliciting an IPv6 router1152vm-test-run-nixbot-gitlab> gitlab # [   10.700120] dhcpcd[1234]: eth0: Router Advertisement from fe80::21153vm-test-run-nixbot-gitlab> gitlab # [   10.701416] dhcpcd[1234]: eth0: adding address fec0::5054:ff:fe12:3456/641154vm-test-run-nixbot-gitlab> gitlab # [   10.702329] dhcpcd[1234]: eth0: adding route to fec0::/641155vm-test-run-nixbot-gitlab> gitlab # [   10.703229] dhcpcd[1234]: eth0: adding default route via fe80::21156vm-test-run-nixbot-gitlab> gitlab # [   10.739666] dhcpcd[1234]: eth0: soliciting a DHCP lease1157vm-test-run-nixbot-gitlab> gitlab # [   11.014759] NET: Registered PF_PACKET protocol family1158vm-test-run-nixbot-gitlab> gitlab # [   10.760990] dhcpcd[1234]: eth0: offered 10.0.2.15 from 10.0.2.21159vm-test-run-nixbot-gitlab> gitlab # [   10.775494] postgresql-setup-start[1345]: CREATE DATABASE1160vm-test-run-nixbot-gitlab> gitlab # [   10.779179] dhcpcd[1234]: eth0: probing address 10.0.2.15/241161vm-test-run-nixbot-gitlab> gitlab # [   10.810513] postgresql-setup-start[1351]: CREATE ROLE1162vm-test-run-nixbot-gitlab> gitlab # [   10.822266] postgresql-setup-start[1357]: ALTER DATABASE1163vm-test-run-nixbot-gitlab> gitlab # [   10.843990] postgresql-setup-start[1363]: CREATE ROLE1164vm-test-run-nixbot-gitlab> gitlab # [   10.846672] systemd[1]: Finished PostgreSQL Setup Scripts.1165vm-test-run-nixbot-gitlab> gitlab # [   10.848526] systemd[1]: Reached target PostgreSQL.1166vm-test-run-nixbot-gitlab> gitlab # [   10.850276] systemd[1]: Starting gitlab-postgresql.service...1167vm-test-run-nixbot-gitlab> gitlab # [   10.946680] 119c5p75zqwmpdrsjsd5cszxwrq2hgfq-gitlab-config[1358]: mkdir -p /var/gitlab/state/home/.ssh: OK1168vm-test-run-nixbot-gitlab> gitlab # [   10.949633] gitlab-postgresql-start[1372]: CREATE DATABASE1169vm-test-run-nixbot-gitlab> gitlab # [   10.959883] 119c5p75zqwmpdrsjsd5cszxwrq2hgfq-gitlab-config[1358]: chmod 700 /var/gitlab/state/home/.ssh: OK1170vm-test-run-nixbot-gitlab> gitlab # [   10.998623] gitlab-postgresql-start[1379]: CREATE EXTENSION1171vm-test-run-nixbot-gitlab> gitlab # [   11.030994] systemd[1]: Finished gitlab-config.service.1172vm-test-run-nixbot-gitlab> gitlab # [   11.036956] gitlab-postgresql-start[1387]: CREATE EXTENSION1173vm-test-run-nixbot-gitlab> gitlab # [   11.043691] systemd[1]: Started gitaly.service.1174vm-test-run-nixbot-gitlab> gitlab # [   11.049653] systemd[1]: Starting gitlab-db-config.service...1175vm-test-run-nixbot-gitlab> gitlab # [   11.054839] systemd[1]: Finished gitlab-postgresql.service.1176vm-test-run-nixbot-gitlab> gitlab # [   11.592693] gitaly[1392]: time="2026-07-04T12:35:52.543Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/2wcq0nlg508fs5wq7jnpg1lw2pib950g-gitaly.toml' and fix the errors that are printed." pid=13921177vm-test-run-nixbot-gitlab> gitlab # [   11.595986] gitaly[1392]: time="2026-07-04T12:35:52.543Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=13921178vm-test-run-nixbot-gitlab> gitlab # [   11.597899] gitaly[1392]: time="2026-07-04T12:35:52.543Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=13921179vm-test-run-nixbot-gitlab> gitlab # [   11.600519] gitaly[1392]: time="2026-07-04T12:35:52.543Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=13921180vm-test-run-nixbot-gitlab> gitlab # [   11.602707] gitaly[1392]: time="2026-07-04T12:35:52.544Z" level=info msg="Starting Gitaly" pid=1392 version=18.11.41181vm-test-run-nixbot-gitlab> gitlab # [   11.819914] gitaly[1392]: time="2026-07-04T12:35:52.771Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=226 pid=13921182vm-test-run-nixbot-gitlab> gitlab # [   11.821467] gitaly[1392]: time="2026-07-04T12:35:52.771Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=13921183vm-test-run-nixbot-gitlab> gitlab # [   11.822256] gitaly[1392]: time="2026-07-04T12:35:52.772Z" level=info msg="finished initializing command factory" duration_ms=0 pid=13921184vm-test-run-nixbot-gitlab> gitlab # [   11.823141] gitaly[1392]: time="2026-07-04T12:35:52.772Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-1392/git-exec-1803879323.d/git pid=13921185vm-test-run-nixbot-gitlab> gitlab # [   11.833487] gitaly[1392]: time="2026-07-04T12:35:52.784Z" level=info msg="finished detecting git version" duration_ms=12 pid=13921186vm-test-run-nixbot-gitlab> gitlab # [   11.835253] gitaly[1392]: time="2026-07-04T12:35:52.785Z" level=info msg="using Git version" pid=1392 version=2.53.g1187vm-test-run-nixbot-gitlab> gitlab # [   11.836442] gitaly[1392]: time="2026-07-04T12:35:52.786Z" level=info msg="clearing disk cache object folder" pid=1392 storage=default1188vm-test-run-nixbot-gitlab> gitlab # [   11.837388] gitaly[1392]: time="2026-07-04T12:35:52.786Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3648969999 pid=1392 storage=default1189vm-test-run-nixbot-gitlab> gitlab # [   11.838452] gitaly[1392]: time="2026-07-04T12:35:52.786Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3648969999 pid=1392 storage=default1190vm-test-run-nixbot-gitlab> gitlab # [   11.839933] gitaly[1392]: time="2026-07-04T12:35:52.786Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=13921191vm-test-run-nixbot-gitlab> gitlab # [   11.840920] gitaly[1392]: time="2026-07-04T12:35:52.787Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=13921192vm-test-run-nixbot-gitlab> gitlab # [   11.842377] gitaly[1392]: time="2026-07-04T12:35:52.787Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3648969999 clear_duration_ms=0 pid=1392 storage=default1193vm-test-run-nixbot-gitlab> gitlab # [   11.908558] gitaly[1392]: time="2026-07-04T12:35:52.860Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=13921194vm-test-run-nixbot-gitlab> gitlab # [   11.909991] gitaly[1392]: time="2026-07-04T12:35:52.860Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=1392 schema=unix1195vm-test-run-nixbot-gitlab> gitlab # [   11.910829] gitaly[1392]: time="2026-07-04T12:35:52.860Z" level=info msg="listening at address" address=/run/gitaly/gitaly-1392/sock.d/intern pid=1392 schema=unix1196vm-test-run-nixbot-gitlab> gitlab # [   11.912114] gitaly[1392]: time="2026-07-04T12:35:52.860Z" level=info msg="finished tempdir cleaner walk" pid=1392 storage=default time_ms=01197vm-test-run-nixbot-gitlab> gitlab # [   11.912919] gitaly[1392]: time="2026-07-04T12:35:52.861Z" level=info msg="counting repositories" pid=13921198vm-test-run-nixbot-gitlab> gitlab # [   11.914122] gitaly[1392]: time="2026-07-04T12:35:52.861Z" level=info msg="starting to count repositories" pid=1392 storage_path=/var/gitlab/state/repositories1199vm-test-run-nixbot-gitlab> gitlab # [   11.915047] gitaly[1392]: time="2026-07-04T12:35:52.861Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=1392 storage_path=/var/gitlab/state/repositories1200vm-test-run-nixbot-gitlab> gitlab # [   11.916249] gitaly[1392]: time="2026-07-04T12:35:52.861Z" level=info msg="completed counting all repositories" counting_duration=0 pid=13921201vm-test-run-nixbot-gitlab> gitlab # [   11.917116] gitaly[1392]: time="2026-07-04T12:35:52.862Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=13921202vm-test-run-nixbot-gitlab> gitlab # [   11.917990] gitaly[1392]: time="2026-07-04T12:35:52.862Z" level=info msg="Started Gitaly" duration_ms=318 pid=13921203vm-test-run-nixbot-gitlab> gitlab # [   11.918866] gitaly[1392]: time="2026-07-04T12:35:52.863Z" level=info msg="maintenance: daily scheduled" pid=1392 scheduled="2026-07-05 12:00:00 +0000 UTC"1204vm-test-run-nixbot-gitlab> gitlab # [   12.507848] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1396]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1205vm-test-run-nixbot-gitlab> gitlab # [   12.510489] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1396]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1206vm-test-run-nixbot-gitlab> gitlab # [   12.552252] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1396]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1207vm-test-run-nixbot-gitlab> gitlab # [   13.824888] gitaly[1392]: time="2026-07-04T12:35:54.776Z" level=info msg="License database preloaded" duration_ms=2231 pid=13921208vm-test-run-nixbot-gitlab> gitlab # [   14.990961] dhcpcd[1234]: eth0: leased 10.0.2.15 for 86400 seconds1209vm-test-run-nixbot-gitlab> gitlab # [   14.993550] dhcpcd[1234]: eth0: adding route to 10.0.2.0/241210vm-test-run-nixbot-gitlab> gitlab # [   14.995363] dhcpcd[1234]: eth0: adding default route via 10.0.2.21211vm-test-run-nixbot-gitlab> gitlab # [   15.129822] systemd[1]: Started DHCP Client.1212vm-test-run-nixbot-gitlab> gitlab # [   15.133280] systemd[1]: Reached target Network is Online.1213vm-test-run-nixbot-gitlab> gitlab # [   44.600574] postgres[1760]: [1760] ERROR:  relation "feature_gates" does not exist at character 5231214vm-test-run-nixbot-gitlab> gitlab # [   44.604103] postgres[1760]: [1760] STATEMENT:  SELECT a.attname, format_type(a.atttypid, a.atttypmod),1215vm-test-run-nixbot-gitlab> gitlab # [   44.605956] postgres[1760]: 	       pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1216vm-test-run-nixbot-gitlab> gitlab # [   44.610852] postgres[1760]: 	       c.collname, col_description(a.attrelid, a.attnum) AS comment,1217vm-test-run-nixbot-gitlab> gitlab # [   44.615065] postgres[1760]: 	       attidentity AS identity,1218vm-test-run-nixbot-gitlab> gitlab # [   44.617142] postgres[1760]: 	       attgenerated as attgenerated1219vm-test-run-nixbot-gitlab> gitlab # [   44.620596] postgres[1760]: 	  FROM pg_attribute a1220vm-test-run-nixbot-gitlab> gitlab # [   44.623953] postgres[1760]: 	  LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1221vm-test-run-nixbot-gitlab> gitlab # [   44.628131] postgres[1760]: 	  LEFT JOIN pg_type t ON a.atttypid = t.oid1222vm-test-run-nixbot-gitlab> gitlab # [   44.632152] postgres[1760]: 	  LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1223vm-test-run-nixbot-gitlab> gitlab # [   44.635561] postgres[1760]: 	 WHERE a.attrelid = '"feature_gates"'::regclass1224vm-test-run-nixbot-gitlab> gitlab # [   44.638956] postgres[1760]: 	   AND a.attnum > 0 AND NOT a.attisdropped1225vm-test-run-nixbot-gitlab> gitlab # [   44.643077] postgres[1760]: 	 ORDER BY a.attnum1226vm-test-run-nixbot-gitlab> gitlab # [   44.645799] postgres[1760]:1227vm-test-run-nixbot-gitlab> gitlab # [   44.856946] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1396]: W, [2026-07-04T12:36:25.807374 #1396]  WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1228vm-test-run-nixbot-gitlab> gitlab # [   49.575293] gitlab-workhorse[1293]: time="2026-07-04T12:36:30Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1229vm-test-run-nixbot-gitlab> gitlab # [   50.169182] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1815]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE:  extension "btree_gist" already exists, skipping1230vm-test-run-nixbot-gitlab> gitlab # [   50.177747] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1815]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE:  extension "pg_trgm" already exists, skipping1231vm-test-run-nixbot-gitlab> gitlab # [   59.576224] gitlab-workhorse[1293]: time="2026-07-04T12:36:40Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1232vm-test-run-nixbot-gitlab> gitlab # [   68.059444] postgres[1899]: [1899] ERROR:  relation "table_without_models" does not exist1233vm-test-run-nixbot-gitlab> gitlab # [   68.062837] postgres[1899]: [1899] STATEMENT:  SELECT pg_get_serial_sequence('table_without_models', 'id')1234vm-test-run-nixbot-gitlab> gitlab # [   69.586306] gitlab-workhorse[1293]: time="2026-07-04T12:36:50Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1235vm-test-run-nixbot-gitlab> gitlab # [   79.586849] gitlab-workhorse[1293]: time="2026-07-04T12:37:00Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1236vm-test-run-nixbot-gitlab> gitlab # [   84.177701] systemd[1]: Finished gitlab-db-config.service.1237vm-test-run-nixbot-gitlab> gitlab # [   84.186920] systemd[1]: Started gitlab-sidekiq.service.1238vm-test-run-nixbot-gitlab> gitlab # [   84.194303] systemd[1]: Starting gitlab.service...1239vm-test-run-nixbot-gitlab> gitlab # [   84.563401] bundle[2120]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1240vm-test-run-nixbot-gitlab> gitlab # [   84.567229] bundle[2120]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1241vm-test-run-nixbot-gitlab> gitlab # [   84.570477] bundle[2120]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1242vm-test-run-nixbot-gitlab> gitlab # [   87.582326] sidekiq-cluster[2119]: {"severity":"INFO","time":"2026-07-04T12:37:08.514Z","message":"Starting cluster with 1 processes"}1243vm-test-run-nixbot-gitlab> gitlab # [   87.838157] bundle[2120]: {"timestamp":"2026-07-04T12:37:08.765Z","pid":2120,"message":"* Enabling systemd notification integration"}1244vm-test-run-nixbot-gitlab> gitlab # [   87.840471] bundle[2120]: {"timestamp":"2026-07-04T12:37:08.788Z","pid":2120,"message":"Puma starting in cluster mode..."}1245vm-test-run-nixbot-gitlab> gitlab # [   87.841991] bundle[2120]: {"timestamp":"2026-07-04T12:37:08.788Z","pid":2120,"message":"* Puma version: 7.2.0 (\"On The Corner\")"}1246vm-test-run-nixbot-gitlab> gitlab # [   87.843728] bundle[2120]: {"timestamp":"2026-07-04T12:37:08.789Z","pid":2120,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1247vm-test-run-nixbot-gitlab> gitlab # [   87.845328] bundle[2120]: {"timestamp":"2026-07-04T12:37:08.789Z","pid":2120,"message":"*  Min threads: 0"}1248vm-test-run-nixbot-gitlab> gitlab # [   87.846542] bundle[2120]: {"timestamp":"2026-07-04T12:37:08.789Z","pid":2120,"message":"*  Max threads: 4"}1249vm-test-run-nixbot-gitlab> gitlab # [   87.848633] bundle[2120]: {"timestamp":"2026-07-04T12:37:08.789Z","pid":2120,"message":"*  Environment: production"}1250vm-test-run-nixbot-gitlab> gitlab # [   87.850399] bundle[2120]: {"timestamp":"2026-07-04T12:37:08.789Z","pid":2120,"message":"*   Master PID: 2120"}1251vm-test-run-nixbot-gitlab> gitlab # [   87.851872] bundle[2120]: {"timestamp":"2026-07-04T12:37:08.789Z","pid":2120,"message":"*      Workers: 2"}1252vm-test-run-nixbot-gitlab> gitlab # [   87.853464] bundle[2120]: {"timestamp":"2026-07-04T12:37:08.789Z","pid":2120,"message":"*     Restarts: (✔) hot (✖) phased (✖) refork"}1253vm-test-run-nixbot-gitlab> gitlab # [   87.854922] bundle[2120]: {"timestamp":"2026-07-04T12:37:08.789Z","pid":2120,"message":"* Preloading application"}1254vm-test-run-nixbot-gitlab> gitlab # [   89.597504] gitlab-workhorse[1293]: time="2026-07-04T12:37:10Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1255vm-test-run-nixbot-gitlab> gitlab # [   99.612427] gitlab-workhorse[1293]: time="2026-07-04T12:37:20Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1256vm-test-run-nixbot-gitlab> gitlab # [  109.614179] gitlab-workhorse[1293]: time="2026-07-04T12:37:30Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1257vm-test-run-nixbot-gitlab> gitlab # [  112.893633] bundle[2120]: W, [2026-07-04T12:37:33.844239 #2120]  WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1258vm-test-run-nixbot-gitlab> gitlab # [  117.400991] sidekiq-cluster[2150]: 2026-07-04T12:37:38.351Z pid=2150 tid=u0e INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2150, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>431607808}1259vm-test-run-nixbot-gitlab> gitlab # [  117.599115] sidekiq-cluster[2150]: W, [2026-07-04T12:37:38.550354 #2150]  WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1260vm-test-run-nixbot-gitlab> gitlab # [  117.642900] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:38.594Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1261vm-test-run-nixbot-gitlab> gitlab # [  119.615949] gitlab-workhorse[1293]: time="2026-07-04T12:37:40Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1262vm-test-run-nixbot-gitlab> gitlab # [  119.760195] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.710Z","message":"GitLab reliable fetch activated!"}1263vm-test-run-nixbot-gitlab> gitlab # [  119.761997] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.710Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1264vm-test-run-nixbot-gitlab> gitlab # [  119.821669] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.773Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1265vm-test-run-nixbot-gitlab> gitlab # [  119.830921] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.782Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1266vm-test-run-nixbot-gitlab> gitlab # [  119.835265] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.786Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1267vm-test-run-nixbot-gitlab> gitlab # [  119.839253] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.790Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1268vm-test-run-nixbot-gitlab> gitlab # [  119.843219] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.794Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1269vm-test-run-nixbot-gitlab> gitlab # [  119.846979] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.798Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1270vm-test-run-nixbot-gitlab> gitlab # [  119.850718] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.802Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1271vm-test-run-nixbot-gitlab> gitlab # [  119.854217] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.805Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1272vm-test-run-nixbot-gitlab> gitlab # [  119.857683] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.809Z","message":"Cron Jobs - added job with name expire_build_artifacts_worker in the namespace default"}1273vm-test-run-nixbot-gitlab> gitlab # [  119.861242] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.812Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1274vm-test-run-nixbot-gitlab> gitlab # [  119.864691] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.816Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1275vm-test-run-nixbot-gitlab> gitlab # [  119.869125] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.820Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1276vm-test-run-nixbot-gitlab> gitlab # [  119.872342] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.823Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1277vm-test-run-nixbot-gitlab> gitlab # [  119.875739] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.827Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1278vm-test-run-nixbot-gitlab> gitlab # [  119.879216] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.830Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1279vm-test-run-nixbot-gitlab> gitlab # [  119.883427] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.834Z","message":"Cron Jobs - added job with name environments_auto_delete_cron_worker in the namespace default"}1280vm-test-run-nixbot-gitlab> gitlab # [  119.887780] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.839Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1281vm-test-run-nixbot-gitlab> gitlab # [  119.892791] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.844Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1282vm-test-run-nixbot-gitlab> gitlab # [  119.896220] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.847Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1283vm-test-run-nixbot-gitlab> gitlab # [  119.898997] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.850Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1284vm-test-run-nixbot-gitlab> gitlab # [  119.901856] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.853Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1285vm-test-run-nixbot-gitlab> gitlab # [  119.906851] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.856Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1286vm-test-run-nixbot-gitlab> gitlab # [  119.911159] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.859Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1287vm-test-run-nixbot-gitlab> gitlab # [  119.914962] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.861Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1288vm-test-run-nixbot-gitlab> gitlab # [  119.916034] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.863Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1289vm-test-run-nixbot-gitlab> gitlab # [  119.916813] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.866Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1290vm-test-run-nixbot-gitlab> gitlab # [  119.918038] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.869Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1291vm-test-run-nixbot-gitlab> gitlab # [  119.920429] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.872Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1292vm-test-run-nixbot-gitlab> gitlab # [  119.923469] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.874Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1293vm-test-run-nixbot-gitlab> gitlab # [  119.926275] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.877Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1294vm-test-run-nixbot-gitlab> gitlab # [  119.928847] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.880Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1295vm-test-run-nixbot-gitlab> gitlab # [  119.931523] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.883Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1296vm-test-run-nixbot-gitlab> gitlab # [  119.934794] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.886Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1297vm-test-run-nixbot-gitlab> gitlab # [  119.937449] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.889Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1298vm-test-run-nixbot-gitlab> gitlab # [  119.940038] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.891Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1299vm-test-run-nixbot-gitlab> gitlab # [  119.942496] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.894Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1300vm-test-run-nixbot-gitlab> gitlab # [  119.945277] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.896Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1301vm-test-run-nixbot-gitlab> gitlab # [  119.947532] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.899Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1302vm-test-run-nixbot-gitlab> gitlab # [  119.950267] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.901Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1303vm-test-run-nixbot-gitlab> gitlab # [  119.952885] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.904Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1304vm-test-run-nixbot-gitlab> gitlab # [  119.955235] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.906Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1305vm-test-run-nixbot-gitlab> gitlab # [  119.957716] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.909Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1306vm-test-run-nixbot-gitlab> gitlab # [  119.960167] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.911Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1307vm-test-run-nixbot-gitlab> gitlab # [  119.962620] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.914Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1308vm-test-run-nixbot-gitlab> gitlab # [  119.965517] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.916Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1309vm-test-run-nixbot-gitlab> gitlab # [  119.968259] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.919Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1310vm-test-run-nixbot-gitlab> gitlab # [  119.971071] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.922Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1311vm-test-run-nixbot-gitlab> gitlab # [  119.973685] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.925Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1312vm-test-run-nixbot-gitlab> gitlab # [  119.976179] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.927Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1313vm-test-run-nixbot-gitlab> gitlab # [  119.979145] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.930Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1314vm-test-run-nixbot-gitlab> gitlab # [  119.981859] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.933Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1315vm-test-run-nixbot-gitlab> gitlab # [  119.984671] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.936Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1316vm-test-run-nixbot-gitlab> gitlab # [  119.987667] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.939Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1317vm-test-run-nixbot-gitlab> gitlab # [  119.997142] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.948Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1318vm-test-run-nixbot-gitlab> gitlab # [  120.000317] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.951Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1319vm-test-run-nixbot-gitlab> gitlab # [  120.002888] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.954Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1320vm-test-run-nixbot-gitlab> gitlab # [  120.005957] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.957Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1321vm-test-run-nixbot-gitlab> gitlab # [  120.008769] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.960Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1322vm-test-run-nixbot-gitlab> gitlab # [  120.011872] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.963Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1323vm-test-run-nixbot-gitlab> gitlab # [  120.014271] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.965Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1324vm-test-run-nixbot-gitlab> gitlab # [  120.016986] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.968Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1325vm-test-run-nixbot-gitlab> gitlab # [  120.019741] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.971Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1326vm-test-run-nixbot-gitlab> gitlab # [  120.022208] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.973Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1327vm-test-run-nixbot-gitlab> gitlab # [  120.025391] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.977Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1328vm-test-run-nixbot-gitlab> gitlab # [  120.028606] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.980Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1329vm-test-run-nixbot-gitlab> gitlab # [  120.031786] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.983Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1330vm-test-run-nixbot-gitlab> gitlab # [  120.034876] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.986Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1331vm-test-run-nixbot-gitlab> gitlab # [  120.037361] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.989Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1332vm-test-run-nixbot-gitlab> gitlab # [  120.039804] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.991Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1333vm-test-run-nixbot-gitlab> gitlab # [  120.042373] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:40.994Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1334vm-test-run-nixbot-gitlab> gitlab # [  120.068961] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.020Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1335vm-test-run-nixbot-gitlab> gitlab # [  120.071213] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.022Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1336vm-test-run-nixbot-gitlab> gitlab # [  120.073537] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.025Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1337vm-test-run-nixbot-gitlab> gitlab # [  120.076102] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.027Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1338vm-test-run-nixbot-gitlab> gitlab # [  120.079112] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.030Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1339vm-test-run-nixbot-gitlab> gitlab # [  120.081368] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.032Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1340vm-test-run-nixbot-gitlab> gitlab # [  120.084578] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.036Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1341vm-test-run-nixbot-gitlab> gitlab # [  120.087103] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.038Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1342vm-test-run-nixbot-gitlab> gitlab # [  120.090173] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.041Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1343vm-test-run-nixbot-gitlab> gitlab # [  120.092881] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.044Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1344vm-test-run-nixbot-gitlab> gitlab # [  120.096735] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.048Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1345vm-test-run-nixbot-gitlab> gitlab # [  120.100240] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.051Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1346vm-test-run-nixbot-gitlab> gitlab # [  120.102707] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.054Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1347vm-test-run-nixbot-gitlab> gitlab # [  120.106391] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.057Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1348vm-test-run-nixbot-gitlab> gitlab # [  120.108718] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.060Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1349vm-test-run-nixbot-gitlab> gitlab # [  120.111760] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.063Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1350vm-test-run-nixbot-gitlab> gitlab # [  120.114090] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.065Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1351vm-test-run-nixbot-gitlab> gitlab # [  120.117533] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.069Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1352vm-test-run-nixbot-gitlab> gitlab # [  120.119998] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.071Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1353vm-test-run-nixbot-gitlab> gitlab # [  120.122966] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.074Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1354vm-test-run-nixbot-gitlab> gitlab # [  120.125322] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.076Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1355vm-test-run-nixbot-gitlab> gitlab # [  120.128618] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.080Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1356vm-test-run-nixbot-gitlab> gitlab # [  120.130837] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.082Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1357vm-test-run-nixbot-gitlab> gitlab # [  120.133900] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.085Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1358vm-test-run-nixbot-gitlab> gitlab # [  120.136355] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.087Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1359vm-test-run-nixbot-gitlab> gitlab # [  120.139749] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.091Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1360vm-test-run-nixbot-gitlab> gitlab # [  120.142502] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.094Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1361vm-test-run-nixbot-gitlab> gitlab # [  120.145189] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.096Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1362vm-test-run-nixbot-gitlab> gitlab # [  120.147669] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.099Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1363vm-test-run-nixbot-gitlab> gitlab # [  120.156345] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.107Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1364vm-test-run-nixbot-gitlab> gitlab # [  120.159166] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.110Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1365vm-test-run-nixbot-gitlab> gitlab # [  120.161443] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.113Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1366vm-test-run-nixbot-gitlab> gitlab # [  120.164612] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.116Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1367vm-test-run-nixbot-gitlab> gitlab # [  120.167841] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.119Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1368vm-test-run-nixbot-gitlab> gitlab # [  120.170433] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.122Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1369vm-test-run-nixbot-gitlab> gitlab # [  120.173391] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.124Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1370vm-test-run-nixbot-gitlab> gitlab # [  120.176774] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.128Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1371vm-test-run-nixbot-gitlab> gitlab # [  120.182376] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.133Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1372vm-test-run-nixbot-gitlab> gitlab # [  120.186606] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.138Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1373vm-test-run-nixbot-gitlab> gitlab # [  120.189332] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:37:41.140Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1374vm-test-run-nixbot-gitlab> gitlab # [  129.621877] gitlab-workhorse[1293]: time="2026-07-04T12:37:50Z" level=error correlation_id= duration_ms=2 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1375vm-test-run-nixbot-gitlab> gitlab # [  139.626274] gitlab-workhorse[1293]: time="2026-07-04T12:38:00Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1376vm-test-run-nixbot-gitlab> gitlab # [  141.626650] bundle[2120]: {"timestamp":"2026-07-04T12:38:02.577Z","pid":2120,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1377vm-test-run-nixbot-gitlab> gitlab # [  141.628751] bundle[2120]: {"timestamp":"2026-07-04T12:38:02.577Z","pid":2120,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1378vm-test-run-nixbot-gitlab> gitlab # [  141.630390] bundle[2120]: {"timestamp":"2026-07-04T12:38:02.577Z","pid":2120,"message":"! #\u003cThread:0x000072cf2c4a1e88 /nix/store/g5qraa51hgg5fl7n3da89ygf00qfv95x-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/g5qraa51hgg5fl7n3da89ygf00qfv95x-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1379vm-test-run-nixbot-gitlab> gitlab # [  141.632298] bundle[2120]: {"timestamp":"2026-07-04T12:38:02.577Z","pid":2120,"message":"! #\u003cThread:0x000072cf2c6ee268 /nix/store/wg5r0zmy3rh1kga7z4z7mbs298p0wxli-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/wg5r0zmy3rh1kga7z4z7mbs298p0wxli-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1380vm-test-run-nixbot-gitlab> gitlab # [  141.633655] bundle[2120]: {"timestamp":"2026-07-04T12:38:02.577Z","pid":2120,"message":"Use Ctrl-C to stop"}1381vm-test-run-nixbot-gitlab> gitlab # [  141.635388] bundle[2120]: {"timestamp":"2026-07-04T12:38:02.578Z","pid":2120,"message":"Sending status to systemd every 1.0 sec"}1382vm-test-run-nixbot-gitlab> gitlab # [  142.096137] bundle[2120]: {"timestamp":"2026-07-04T12:38:03.047Z","pid":2120,"message":"- Worker 1 (PID: 2663) booted in 0.37s, phase: 0"}1383vm-test-run-nixbot-gitlab> gitlab # [  142.128661] bundle[2120]: {"timestamp":"2026-07-04T12:38:03.079Z","pid":2120,"message":"- Worker 0 (PID: 2659) booted in 0.43s, phase: 0"}1384vm-test-run-nixbot-gitlab> gitlab # [  142.135640] systemd[1]: Started gitlab.service.1385vm-test-run-nixbot-gitlab> gitlab # [  142.143702] systemd[1]: Reached target Common target for all GitLab services..1386vm-test-run-nixbot-gitlab> gitlab # [  142.147257] systemd[1]: Starting setup-gitlab.service...1387vm-test-run-nixbot-gitlab> gitlab # [  142.171929] setup-gitlab-start[2696]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1388vm-test-run-nixbot-gitlab> gitlab # [  143.257626] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:04.207Z","message":"Booted Rails 7.2.3 application in production environment"}1389vm-test-run-nixbot-gitlab> gitlab # [  143.259851] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:04.209Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1390vm-test-run-nixbot-gitlab> gitlab # [  143.261548] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:04.209Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1391vm-test-run-nixbot-gitlab> gitlab # [  143.263229] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:04.209Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1392vm-test-run-nixbot-gitlab> gitlab # [  143.272616] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:04.223Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1393vm-test-run-nixbot-gitlab> gitlab # [  144.659772] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:38:05 +0000] "GET /users/sign_in HTTP/1.1" 200 10624 "" "curl/8.20.0" 24511394vm-test-run-nixbot-gitlab> gitlab # [  144.667360] setup-gitlab-start[2717]: ++ curl -fs -X POST -H 'Content-Type: application/json' -d '{"grant_type":"password","username":"root","password":"notproduction"}' http://gitlab/oauth/token1395vm-test-run-nixbot-gitlab> gitlab # [  144.669772] setup-gitlab-start[2718]: ++ jq -r .access_token1396vm-test-run-nixbot-gitlab> gitlab # [  144.891704] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:05.842Z","message":"Cleaning working queues"}1397vm-test-run-nixbot-gitlab> gitlab # [  144.895504] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:05.846Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"d33fdb430d6719ec003eb754","created_at":"2026-07-04T12:37:04.474Z","trace_propagation_headers":{"sentry-trace":"eebc49706ca440709ef5959a3a9d42d1-a4963dbf9e3a4f66","baggage":"sentry-trace_id=eebc49706ca440709ef5959a3a9d42d1,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"2d4443499126635a95cfea68bf4c373a","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-04T12:37:04.484Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-d33fdb430d6719ec003eb754: start","job_status":"start","queue_duration_s":61.361781,"scheduling_latency_s":61.361887}1398vm-test-run-nixbot-gitlab> gitlab # [  145.135679] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:06.086Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"d33fdb430d6719ec003eb754","created_at":"2026-07-04T12:37:04.474Z","trace_propagation_headers":{"sentry-trace":"eebc49706ca440709ef5959a3a9d42d1-a4963dbf9e3a4f66","baggage":"sentry-trace_id=eebc49706ca440709ef5959a3a9d42d1,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"2d4443499126635a95cfea68bf4c373a","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-04T12:37:04.484Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-d33fdb430d6719ec003eb754: done: 0.240294 sec","job_status":"done","queue_duration_s":61.361781,"scheduling_latency_s":61.361887,"redis_calls":26,"redis_duration_s":0.015173,"redis_read_bytes":914,"redis_write_bytes":2887,"redis_feature_flag_calls":15,"redis_feature_flag_duration_s":0.007891,"redis_feature_flag_read_bytes":904,"redis_feature_flag_write_bytes":1640,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.004622,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00266,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.113537,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.104001765,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.240294,"completed_at":"2026-07-04T12:38:06.086Z","load_balancing_strategy":"primary","db_duration_s":0.061767,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1399vm-test-run-nixbot-gitlab> gitlab # [  145.543419] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:38:06 +0000] "POST /oauth/token HTTP/1.1" 200 244 "" "curl/8.20.0" 8491400vm-test-run-nixbot-gitlab> gitlab # [  145.554780] setup-gitlab-start[2696]: + BEARER=7cabaaef6f9590979a7fef01ec7730d86857a6110808247748a0c12c4730a6ac1401vm-test-run-nixbot-gitlab> gitlab # [  145.557993] setup-gitlab-start[2696]: + AUTH='Authorization: Bearer 7cabaaef6f9590979a7fef01ec7730d86857a6110808247748a0c12c4730a6ac'1402vm-test-run-nixbot-gitlab> gitlab # [  145.560882] setup-gitlab-start[2696]: + curl -fs -X PUT -H 'Authorization: Bearer 7cabaaef6f9590979a7fef01ec7730d86857a6110808247748a0c12c4730a6ac' 'http://gitlab/api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true'1403vm-test-run-nixbot-gitlab> gitlab # [  146.283398] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:38:07 +0000] "PUT /api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true HTTP/1.1" 200 17133 "" "curl/8.20.0" 7061404vm-test-run-nixbot-gitlab> gitlab # [  146.295828] setup-gitlab-start[2696]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner Rails.cache.clear1405vm-test-run-nixbot-gitlab> gitlab # [  146.317125] runuser[2743]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1406vm-test-run-nixbot-gitlab> gitlab # [  146.669647] setup-gitlab-start[2744]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1407vm-test-run-nixbot-gitlab> gitlab # [  146.673164] setup-gitlab-start[2744]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1408vm-test-run-nixbot-gitlab> gitlab # [  146.675256] setup-gitlab-start[2744]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1409vm-test-run-nixbot-gitlab> gitlab # [  147.270331] setup-gitlab-start[2744]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1410vm-test-run-nixbot-gitlab> gitlab # [  147.273798] setup-gitlab-start[2744]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1411vm-test-run-nixbot-gitlab> gitlab # [  147.276413] setup-gitlab-start[2744]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1412vm-test-run-nixbot-gitlab> gitlab # [  159.576486] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:20.526Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"46846acc1279ab1f024be8f0","created_at":"2026-07-04T12:38:20.500Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3188ad275670773572549e47bd7136d3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:20.522Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-46846acc1279ab1f024be8f0: start","job_status":"start","queue_duration_s":0.003669,"scheduling_latency_s":0.003772}1413vm-test-run-nixbot-gitlab> gitlab # [  159.632810] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:20.584Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"46846acc1279ab1f024be8f0","created_at":"2026-07-04T12:38:20.500Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3188ad275670773572549e47bd7136d3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:20.522Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-46846acc1279ab1f024be8f0: done: 0.057895 sec","job_status":"done","queue_duration_s":0.003669,"scheduling_latency_s":0.003772,"redis_calls":12,"redis_duration_s":0.01096,"redis_read_bytes":11,"redis_write_bytes":1482,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.005122,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003459,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002379,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016221,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000716851,"duration_s":0.057895,"completed_at":"2026-07-04T12:38:20.583Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008483,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1414vm-test-run-nixbot-gitlab> gitlab # [  159.674647] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:20.594Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"5c2f7d4f09d9df9401b46cb8","created_at":"2026-07-04T12:38:20.532Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"68a72a28ce0e32b8e65280ee706502dd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:20.545Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-5c2f7d4f09d9df9401b46cb8: start","job_status":"start","queue_duration_s":0.049107,"scheduling_latency_s":0.049133}1415vm-test-run-nixbot-gitlab> gitlab # [  159.768759] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:20.719Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"5c2f7d4f09d9df9401b46cb8","created_at":"2026-07-04T12:38:20.532Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"68a72a28ce0e32b8e65280ee706502dd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:20.545Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-5c2f7d4f09d9df9401b46cb8: done: 0.124481 sec","job_status":"done","queue_duration_s":0.049107,"scheduling_latency_s":0.049133,"redis_calls":10,"redis_duration_s":0.063588,"redis_read_bytes":9,"redis_write_bytes":1299,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.054062,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.007651,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001875,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02274,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001731505,"duration_s":0.124481,"completed_at":"2026-07-04T12:38:20.719Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002149,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1416vm-test-run-nixbot-gitlab> gitlab # [  159.843548] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:20.794Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"637e343ec4ca58715d6d5303","created_at":"2026-07-04T12:38:20.561Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1dac8bc75c4950901518fdac7cd3ca36","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:20.572Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-637e343ec4ca58715d6d5303: start","job_status":"start","queue_duration_s":0.222364,"scheduling_latency_s":0.222438}1417vm-test-run-nixbot-gitlab> gitlab # [  159.918717] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:20.870Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1418vm-test-run-nixbot-gitlab> gitlab # [  159.943172] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:20.894Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"637e343ec4ca58715d6d5303","created_at":"2026-07-04T12:38:20.561Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1dac8bc75c4950901518fdac7cd3ca36","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:20.572Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-637e343ec4ca58715d6d5303: done: 0.099863 sec","job_status":"done","queue_duration_s":0.222364,"scheduling_latency_s":0.222438,"redis_calls":10,"redis_duration_s":0.017372,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.007131,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.008784,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001457,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0257,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.002574908,"duration_s":0.099863,"completed_at":"2026-07-04T12:38:20.894Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004064,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1419vm-test-run-nixbot-gitlab> gitlab # [  159.986271] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:20.898Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"2f2e4ba7f1353e88b9051c3e","created_at":"2026-07-04T12:38:20.593Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3ce9961fa7cbba84928d8c2ee847020d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:20.605Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-2f2e4ba7f1353e88b9051c3e: start","job_status":"start","queue_duration_s":0.29286,"scheduling_latency_s":0.292905}1420vm-test-run-nixbot-gitlab> gitlab # [  160.704238] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:21.654Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"2f2e4ba7f1353e88b9051c3e","created_at":"2026-07-04T12:38:20.593Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3ce9961fa7cbba84928d8c2ee847020d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:20.605Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-2f2e4ba7f1353e88b9051c3e: done: 0.755811 sec","job_status":"done","queue_duration_s":0.29286,"scheduling_latency_s":0.292905,"redis_calls":14,"redis_duration_s":0.0068839999999999995,"redis_read_bytes":13,"redis_write_bytes":1645,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001416,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002646,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002822,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":125,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":125,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.065,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":125,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":80,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.065,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.431089,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001718654,"exclusive_lock_hold_duration_s":0.654973187,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.755811,"completed_at":"2026-07-04T12:38:21.654Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.104157,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1421vm-test-run-nixbot-gitlab> gitlab # [  160.807947] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:21.759Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"50912a0ef955ed2b352818b4","created_at":"2026-07-04T12:38:20.619Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"367bc4a615d210d2c9d5aeb4bc27a5fe","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:20.680Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-50912a0ef955ed2b352818b4: start","job_status":"start","queue_duration_s":1.078427,"scheduling_latency_s":1.078475}1422vm-test-run-nixbot-gitlab> gitlab # [  160.888849] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:21.839Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"50912a0ef955ed2b352818b4","created_at":"2026-07-04T12:38:20.619Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"367bc4a615d210d2c9d5aeb4bc27a5fe","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:20.680Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-50912a0ef955ed2b352818b4: done: 0.080494 sec","job_status":"done","queue_duration_s":1.078427,"scheduling_latency_s":1.078475,"redis_calls":12,"redis_duration_s":0.005089,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000856,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002541,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001692,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0336,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00106522,"exclusive_lock_hold_duration_s":0.009029919,"duration_s":0.080494,"completed_at":"2026-07-04T12:38:21.839Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005669,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1423vm-test-run-nixbot-gitlab> gitlab # [  160.969107] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:21.919Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"6d293d76a14413d9642e1f45","created_at":"2026-07-04T12:38:20.708Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a71dc1c20d7aa2e2d93153c356287df8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:20.790Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-6d293d76a14413d9642e1f45: start","job_status":"start","queue_duration_s":1.129395,"scheduling_latency_s":1.129467}1424vm-test-run-nixbot-gitlab> gitlab # [  161.059526] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:22.010Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"6d293d76a14413d9642e1f45","created_at":"2026-07-04T12:38:20.708Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a71dc1c20d7aa2e2d93153c356287df8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:20.790Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-6d293d76a14413d9642e1f45: done: 0.090528 sec","job_status":"done","queue_duration_s":1.129395,"scheduling_latency_s":1.129467,"redis_calls":12,"redis_duration_s":0.00894,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001446,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004646,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002848,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.046113,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001457168,"exclusive_lock_hold_duration_s":0.021353831,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.090528,"completed_at":"2026-07-04T12:38:22.009Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.023656,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1425vm-test-run-nixbot-gitlab> gitlab # [  161.064597] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:22.013Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"57ab3b4e105a194a709ab6e6","created_at":"2026-07-04T12:38:20.822Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d11c59fdf9b72e0635bcc4862203b2d8","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:20.834Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-57ab3b4e105a194a709ab6e6: start","job_status":"start","queue_duration_s":1.17899,"scheduling_latency_s":1.179031}1426vm-test-run-nixbot-gitlab> gitlab # [  161.692448] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:22.643Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"57ab3b4e105a194a709ab6e6","created_at":"2026-07-04T12:38:20.822Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d11c59fdf9b72e0635bcc4862203b2d8","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:20.834Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-57ab3b4e105a194a709ab6e6: done: 0.630206 sec","job_status":"done","queue_duration_s":1.17899,"scheduling_latency_s":1.179031,"redis_calls":574,"redis_duration_s":0.18433,"redis_read_bytes":573,"redis_write_bytes":49866,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000963,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002274,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.181093,"redis_shared_state_read_bytes":567,"redis_shared_state_write_bytes":49017,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.099,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.099,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.390089,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000621308,"duration_s":0.630206,"completed_at":"2026-07-04T12:38:22.643Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.099272,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1427vm-test-run-nixbot-gitlab> gitlab # [  161.761778] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:22.711Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"519c04143c3f2644bd654619","created_at":"2026-07-04T12:38:20.860Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"72d34d50eff57f49abd91bbcdbd1dbc2","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:20.883Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-519c04143c3f2644bd654619: start","job_status":"start","queue_duration_s":1.82802,"scheduling_latency_s":1.828059}1428vm-test-run-nixbot-gitlab> gitlab # [  161.841990] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:22.792Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"519c04143c3f2644bd654619","created_at":"2026-07-04T12:38:20.860Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"72d34d50eff57f49abd91bbcdbd1dbc2","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:20.883Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-519c04143c3f2644bd654619: done: 0.080706 sec","job_status":"done","queue_duration_s":1.82802,"scheduling_latency_s":1.828059,"redis_calls":12,"redis_duration_s":0.008399,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001299,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003793,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003307,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033334,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001949969,"exclusive_lock_hold_duration_s":0.009844546000000001,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.080706,"completed_at":"2026-07-04T12:38:22.792Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003828,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1429vm-test-run-nixbot-gitlab> gitlab # [  161.915963] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:22.866Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"2a523c87ffe32f705bc76fce","created_at":"2026-07-04T12:38:20.911Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d46172485b0f053c985781b0ee92a991","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:20.945Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-2a523c87ffe32f705bc76fce: start","job_status":"start","queue_duration_s":1.921433,"scheduling_latency_s":1.921504}1430vm-test-run-nixbot-gitlab> gitlab # [  161.972147] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:22.922Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"2a523c87ffe32f705bc76fce","created_at":"2026-07-04T12:38:20.911Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d46172485b0f053c985781b0ee92a991","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:20.945Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-2a523c87ffe32f705bc76fce: done: 0.056315 sec","job_status":"done","queue_duration_s":1.921433,"scheduling_latency_s":1.921504,"redis_calls":10,"redis_duration_s":0.006565,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00128,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003914,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001371,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031921,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000951518,"duration_s":0.056315,"completed_at":"2026-07-04T12:38:22.922Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005139,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1431vm-test-run-nixbot-gitlab> gitlab # [  162.038132] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:22.927Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"2ae4d0ac57a0b3e686697526","created_at":"2026-07-04T12:38:20.998Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5981b5ecb35efb52d458addf2f9665c5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:21.020Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-2ae4d0ac57a0b3e686697526: start","job_status":"start","queue_duration_s":1.907036,"scheduling_latency_s":1.9071}1432vm-test-run-nixbot-gitlab> gitlab # [  162.065124] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:23.015Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"2ae4d0ac57a0b3e686697526","created_at":"2026-07-04T12:38:20.998Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5981b5ecb35efb52d458addf2f9665c5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:21.020Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-2ae4d0ac57a0b3e686697526: done: 0.088899 sec","job_status":"done","queue_duration_s":1.907036,"scheduling_latency_s":1.9071,"redis_calls":19,"redis_duration_s":0.009575,"redis_read_bytes":22,"redis_write_bytes":4785,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001949,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000872,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.005188,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001566,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040287,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000848432,"duration_s":0.088899,"completed_at":"2026-07-04T12:38:23.015Z","load_balancing_strategy":"primary","db_duration_s":0.002677,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1433vm-test-run-nixbot-gitlab> gitlab # [  162.102375] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:23.019Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"efcc60cd899acfbb8e0a772f","created_at":"2026-07-04T12:38:21.031Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:21.039Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-efcc60cd899acfbb8e0a772f: start","job_status":"start","queue_duration_s":1.979628,"scheduling_latency_s":1.979668}1434vm-test-run-nixbot-gitlab> gitlab # [  162.219804] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:23.170Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"efcc60cd899acfbb8e0a772f","created_at":"2026-07-04T12:38:21.031Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:21.039Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-efcc60cd899acfbb8e0a772f: done: 0.151562 sec","job_status":"done","queue_duration_s":1.979628,"scheduling_latency_s":1.979668,"redis_calls":71,"redis_duration_s":0.030174,"redis_read_bytes":120,"redis_write_bytes":42624,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.004524,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.00134,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.023205,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001105,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.113088,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000528279,"duration_s":0.151562,"completed_at":"2026-07-04T12:38:23.170Z","load_balancing_strategy":"primary","db_duration_s":0.003626,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1435vm-test-run-nixbot-gitlab> gitlab # [  162.300437] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:23.251Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"572ff4bea28c8765df904783","created_at":"2026-07-04T12:38:21.053Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cd33a1011ac1c525f5dcc0087ff52866","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:21.060Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-572ff4bea28c8765df904783: start","job_status":"start","queue_duration_s":2.190512,"scheduling_latency_s":2.190583}1436vm-test-run-nixbot-gitlab> gitlab # [  162.358787] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:23.309Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"572ff4bea28c8765df904783","created_at":"2026-07-04T12:38:21.053Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cd33a1011ac1c525f5dcc0087ff52866","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:21.060Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-572ff4bea28c8765df904783: done: 0.058037 sec","job_status":"done","queue_duration_s":2.190512,"scheduling_latency_s":2.190583,"redis_calls":10,"redis_duration_s":0.006406,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001153,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003847,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001406,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032371,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00097694,"duration_s":0.058037,"completed_at":"2026-07-04T12:38:23.309Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006329,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1437vm-test-run-nixbot-gitlab> gitlab # [  162.361740] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:23.312Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"8e48e021d9fdadf7cf72807c","created_at":"2026-07-04T12:38:21.068Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"370e0d77257e18e97fd3237f28f1ee5a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-04T12:38:21.074Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-8e48e021d9fdadf7cf72807c: start","job_status":"start","queue_duration_s":2.237983,"scheduling_latency_s":2.238023}1438vm-test-run-nixbot-gitlab> gitlab # [  162.452979] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:23.404Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1439vm-test-run-nixbot-gitlab> gitlab # [  162.469891] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:23.420Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"8e48e021d9fdadf7cf72807c","created_at":"2026-07-04T12:38:21.068Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"370e0d77257e18e97fd3237f28f1ee5a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-04T12:38:21.074Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-8e48e021d9fdadf7cf72807c: done: 0.107563 sec","job_status":"done","queue_duration_s":2.237983,"scheduling_latency_s":2.238023,"redis_calls":10,"redis_duration_s":0.004189,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001002,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002468,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000719,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.077,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.077,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0165,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001562489,"duration_s":0.107563,"completed_at":"2026-07-04T12:38:23.420Z","load_balancing_strategy":"primary","db_duration_s":0.076828,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1440vm-test-run-nixbot-gitlab> gitlab # [  162.539847] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:23.490Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"8dc863ea1172f40b442d21c4","created_at":"2026-07-04T12:38:21.082Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"453ce1581cd1ce11742de39906d3e015","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:21.088Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-8dc863ea1172f40b442d21c4: start","job_status":"start","queue_duration_s":2.402117,"scheduling_latency_s":2.402187}1441vm-test-run-nixbot-gitlab> gitlab # [  162.571503] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:23.522Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1442vm-test-run-nixbot-gitlab> gitlab # [  162.587179] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:23.537Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"8dc863ea1172f40b442d21c4","created_at":"2026-07-04T12:38:21.082Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"453ce1581cd1ce11742de39906d3e015","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:21.088Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-8dc863ea1172f40b442d21c4: done: 0.047789 sec","job_status":"done","queue_duration_s":2.402117,"scheduling_latency_s":2.402187,"redis_calls":10,"redis_duration_s":0.006101,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001378,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003573,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00115,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025609,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000874413,"duration_s":0.047789,"completed_at":"2026-07-04T12:38:23.537Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001669,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1443vm-test-run-nixbot-gitlab> gitlab # [  162.591567] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:23.541Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"36417f7de0c23f222221f19f","created_at":"2026-07-04T12:38:21.100Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"61227d6243daa421b50ca3f3ca5ba70c","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:21.108Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-36417f7de0c23f222221f19f: start","job_status":"start","queue_duration_s":2.432698,"scheduling_latency_s":2.43274}1444vm-test-run-nixbot-gitlab> gitlab # [  162.717978] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:23.669Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"36417f7de0c23f222221f19f","created_at":"2026-07-04T12:38:21.100Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"61227d6243daa421b50ca3f3ca5ba70c","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:21.108Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-36417f7de0c23f222221f19f: done: 0.127663 sec","job_status":"done","queue_duration_s":2.432698,"scheduling_latency_s":2.43274,"redis_calls":13,"redis_duration_s":0.006093,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000692,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003379,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002022,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.082,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.082,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030263,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.018061513,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.127663,"completed_at":"2026-07-04T12:38:23.668Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.08357,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1445vm-test-run-nixbot-gitlab> gitlab # [  162.755487] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:23.672Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"ded69141772445199a443aea","created_at":"2026-07-04T12:38:21.113Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2daa77eec8da0809eab81b7b408efd57","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:21.120Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-ded69141772445199a443aea: start","job_status":"start","queue_duration_s":2.551217,"scheduling_latency_s":2.55126}1446vm-test-run-nixbot-gitlab> gitlab # [  162.756686] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:23.691Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1447vm-test-run-nixbot-gitlab> gitlab # [  162.757822] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:23.698Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"ded69141772445199a443aea","created_at":"2026-07-04T12:38:21.113Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2daa77eec8da0809eab81b7b408efd57","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:21.120Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-ded69141772445199a443aea: done: 0.026706 sec","job_status":"done","queue_duration_s":2.551217,"scheduling_latency_s":2.55126,"redis_calls":10,"redis_duration_s":0.005002,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000923,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002915,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001164,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013326,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000520178,"duration_s":0.026706,"completed_at":"2026-07-04T12:38:23.698Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000884,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1448vm-test-run-nixbot-gitlab> gitlab # [  162.842751] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:23.700Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"4f5a6fb50aca29b4564ff4da","created_at":"2026-07-04T12:38:21.128Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"80fd590c5a9fee123e3d6589cae82a0f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:21.134Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-4f5a6fb50aca29b4564ff4da: start","job_status":"start","queue_duration_s":2.566009,"scheduling_latency_s":2.566031}1449vm-test-run-nixbot-gitlab> gitlab # [  162.975790] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:23.926Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"4f5a6fb50aca29b4564ff4da","created_at":"2026-07-04T12:38:21.128Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"80fd590c5a9fee123e3d6589cae82a0f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:21.134Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-4f5a6fb50aca29b4564ff4da: done: 0.225316 sec","job_status":"done","queue_duration_s":2.566009,"scheduling_latency_s":2.566031,"redis_calls":14,"redis_duration_s":0.009169,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002355,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003908,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002906,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.158,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.14,"db_ci_duration_s":0.018,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.049823,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001492369,"exclusive_lock_hold_duration_s":0.039224817999999995,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.225316,"completed_at":"2026-07-04T12:38:23.926Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.160956,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1450vm-test-run-nixbot-gitlab> gitlab # [  162.980420] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:23.929Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"3e6cce9bf2581bc6a8730652","created_at":"2026-07-04T12:38:21.139Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f475e5dbfb4127d5f274213e66344cfa","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:21.148Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-3e6cce9bf2581bc6a8730652: start","job_status":"start","queue_duration_s":2.781445,"scheduling_latency_s":2.781487}1451vm-test-run-nixbot-gitlab> gitlab # [  163.126979] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.077Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"3e6cce9bf2581bc6a8730652","created_at":"2026-07-04T12:38:21.139Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f475e5dbfb4127d5f274213e66344cfa","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:21.148Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-3e6cce9bf2581bc6a8730652: done: 0.147435 sec","job_status":"done","queue_duration_s":2.781445,"scheduling_latency_s":2.781487,"redis_calls":12,"redis_duration_s":0.006075,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001089,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00254,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002446,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.098,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.098,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028331,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00140353,"exclusive_lock_hold_duration_s":0.012508851999999999,"duration_s":0.147435,"completed_at":"2026-07-04T12:38:24.077Z","load_balancing_strategy":"primary","db_duration_s":0.101734,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1452vm-test-run-nixbot-gitlab> gitlab # [  163.130262] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.081Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"bba6897f5a1450edca57f0fb","created_at":"2026-07-04T12:38:21.156Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"28cde05b56d4353593c615d063b57dcd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-04T12:38:21.162Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-bba6897f5a1450edca57f0fb: start","job_status":"start","queue_duration_s":2.918956,"scheduling_latency_s":2.919028}1453vm-test-run-nixbot-gitlab> gitlab # [  163.168765] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.119Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"bba6897f5a1450edca57f0fb","created_at":"2026-07-04T12:38:21.156Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"28cde05b56d4353593c615d063b57dcd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-04T12:38:21.162Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-bba6897f5a1450edca57f0fb: done: 0.038757 sec","job_status":"done","queue_duration_s":2.918956,"scheduling_latency_s":2.919028,"redis_calls":10,"redis_duration_s":0.003803,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000697,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002221,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000885,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020286,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000557613,"duration_s":0.038757,"completed_at":"2026-07-04T12:38:24.119Z","load_balancing_strategy":"primary","db_duration_s":0.005536,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1454vm-test-run-nixbot-gitlab> gitlab # [  163.211381] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.126Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"d6c2bd0c1b87f2af2cffd136","created_at":"2026-07-04T12:38:21.168Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a531f9360474071eb446ff7dd7e2edd2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:21.174Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-d6c2bd0c1b87f2af2cffd136: start","job_status":"start","queue_duration_s":2.951872,"scheduling_latency_s":2.951895}1455vm-test-run-nixbot-gitlab> gitlab # [  163.298343] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.145Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1456vm-test-run-nixbot-gitlab> gitlab # [  163.300084] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.154Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"d6c2bd0c1b87f2af2cffd136","created_at":"2026-07-04T12:38:21.168Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a531f9360474071eb446ff7dd7e2edd2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:21.174Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-d6c2bd0c1b87f2af2cffd136: done: 0.028171 sec","job_status":"done","queue_duration_s":2.951872,"scheduling_latency_s":2.951895,"redis_calls":10,"redis_duration_s":0.009496000000000001,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001938,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005227,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002331,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01067,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000831949,"duration_s":0.028171,"completed_at":"2026-07-04T12:38:24.154Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002522,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1457vm-test-run-nixbot-gitlab> gitlab # [  163.307235] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.158Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"0c78d36d3bc8e02d44fd0751","created_at":"2026-07-04T12:38:21.189Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1f0243cd3d419bf369ea6d02f06824ed","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:21.196Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-0c78d36d3bc8e02d44fd0751: start","job_status":"start","queue_duration_s":2.961735,"scheduling_latency_s":2.961757}1458vm-test-run-nixbot-gitlab> gitlab # [  163.393459] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.241Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"0c78d36d3bc8e02d44fd0751","created_at":"2026-07-04T12:38:21.189Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1f0243cd3d419bf369ea6d02f06824ed","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:21.196Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-0c78d36d3bc8e02d44fd0751: done: 0.083251 sec","job_status":"done","queue_duration_s":2.961735,"scheduling_latency_s":2.961757,"redis_calls":16,"redis_duration_s":0.00728,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.003643,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003193,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000444,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.052,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.052,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019305,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000398934,"duration_s":0.083251,"completed_at":"2026-07-04T12:38:24.241Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.053107,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1459vm-test-run-nixbot-gitlab> gitlab # [  163.396076] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.243Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"0fe9cf094c520b35ddbf44b5","created_at":"2026-07-04T12:38:21.202Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"023f40c27c494e75db1a2b09dc5f74ea","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:21.208Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-0fe9cf094c520b35ddbf44b5: start","job_status":"start","queue_duration_s":3.034235,"scheduling_latency_s":3.034258}1460vm-test-run-nixbot-gitlab> gitlab # [  163.416216] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.267Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"0fe9cf094c520b35ddbf44b5","created_at":"2026-07-04T12:38:21.202Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"023f40c27c494e75db1a2b09dc5f74ea","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:21.208Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-0fe9cf094c520b35ddbf44b5: done: 0.024312 sec","job_status":"done","queue_duration_s":3.034235,"scheduling_latency_s":3.034258,"redis_calls":12,"redis_duration_s":0.008244,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.001176,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001619,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004688,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000761,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011075,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001111594,"duration_s":0.024312,"completed_at":"2026-07-04T12:38:24.267Z","load_balancing_strategy":"primary","db_duration_s":0.000759,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1461vm-test-run-nixbot-gitlab> gitlab # [  163.485220] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.271Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"eda887f749370c9782a828dc","created_at":"2026-07-04T12:38:21.214Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d64ef98225529840f0a5b872f9a9da6a","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:21.222Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-eda887f749370c9782a828dc: start","job_status":"start","queue_duration_s":3.049024,"scheduling_latency_s":3.049046}1462vm-test-run-nixbot-gitlab> gitlab # [  163.569519] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.300Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"eda887f749370c9782a828dc","created_at":"2026-07-04T12:38:21.214Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d64ef98225529840f0a5b872f9a9da6a","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-04T12:38:21.222Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-eda887f749370c9782a828dc: done: 0.028956 sec","job_status":"done","queue_duration_s":3.049024,"scheduling_latency_s":3.049046,"redis_calls":10,"redis_duration_s":0.003661,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000437,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001853,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001371,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012584,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000352838,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.028956,"completed_at":"2026-07-04T12:38:24.300Z","load_balancing_strategy":"primary","db_duration_s":0.008064,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1463vm-test-run-nixbot-gitlab> gitlab # [  163.572787] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.301Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"add8a3241a4c342b8faee78f","created_at":"2026-07-04T12:38:21.229Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"72146ba454f6a15ffe76e99a7dbecefc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-04T12:38:21.236Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-add8a3241a4c342b8faee78f: start","job_status":"start","queue_duration_s":3.065401,"scheduling_latency_s":3.065423}1464vm-test-run-nixbot-gitlab> gitlab # [  163.593254] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.312Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1465vm-test-run-nixbot-gitlab> gitlab # [  163.599297] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.318Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"add8a3241a4c342b8faee78f","created_at":"2026-07-04T12:38:21.229Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"72146ba454f6a15ffe76e99a7dbecefc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-04T12:38:21.236Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-add8a3241a4c342b8faee78f: done: 0.017016 sec","job_status":"done","queue_duration_s":3.065401,"scheduling_latency_s":3.065423,"redis_calls":10,"redis_duration_s":0.002244,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000425,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001391,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000428,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01001,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00037826,"duration_s":0.017016,"completed_at":"2026-07-04T12:38:24.318Z","load_balancing_strategy":"primary","db_duration_s":0.000391,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1466vm-test-run-nixbot-gitlab> gitlab # [  163.666107] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.319Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a0c552fcb0198fd182304315","created_at":"2026-07-04T12:38:22.980Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bde3c85a64974027","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"5981b5ecb35efb52d458addf2f9665c5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.002Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a0c552fcb0198fd182304315: start","job_status":"start","queue_duration_s":1.317485,"scheduling_latency_s":1.317505}1467vm-test-run-nixbot-gitlab> gitlab # [  163.686089] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.333Z","message":"No existing merge request to be cleaned up."}1468vm-test-run-nixbot-gitlab> gitlab # [  163.688501] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.344Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a0c552fcb0198fd182304315","created_at":"2026-07-04T12:38:22.980Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bde3c85a64974027","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"5981b5ecb35efb52d458addf2f9665c5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.002Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a0c552fcb0198fd182304315: done: 0.024042 sec","job_status":"done","queue_duration_s":1.317485,"scheduling_latency_s":1.317505,"redis_calls":13,"redis_duration_s":0.003106,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000468,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001163,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001475,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015018,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000343339,"duration_s":0.024042,"completed_at":"2026-07-04T12:38:24.343Z","load_balancing_strategy":"primary","db_duration_s":0.003072,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1469vm-test-run-nixbot-gitlab> gitlab # [  163.776932] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.349Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d6ca86290ae5e6d9e6af584d","created_at":"2026-07-04T12:38:22.980Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bde3c85a64974027","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"5981b5ecb35efb52d458addf2f9665c5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.002Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d6ca86290ae5e6d9e6af584d: start","job_status":"start","queue_duration_s":1.34674,"scheduling_latency_s":1.346763}1470vm-test-run-nixbot-gitlab> gitlab # [  163.778216] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.363Z","message":"No existing merge request to be cleaned up."}1471vm-test-run-nixbot-gitlab> gitlab # [  163.779183] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.373Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d6ca86290ae5e6d9e6af584d","created_at":"2026-07-04T12:38:22.980Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bde3c85a64974027","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"5981b5ecb35efb52d458addf2f9665c5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.002Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d6ca86290ae5e6d9e6af584d: done: 0.024449 sec","job_status":"done","queue_duration_s":1.34674,"scheduling_latency_s":1.346763,"redis_calls":10,"redis_duration_s":0.006670000000000001,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00248,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00419,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011127,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024449,"completed_at":"2026-07-04T12:38:24.373Z","load_balancing_strategy":"primary","db_duration_s":0.003037,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1472vm-test-run-nixbot-gitlab> gitlab # [  163.836331] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.375Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8ab77bedec9433f002f68850","created_at":"2026-07-04T12:38:22.980Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bde3c85a64974027","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"5981b5ecb35efb52d458addf2f9665c5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.002Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8ab77bedec9433f002f68850: start","job_status":"start","queue_duration_s":1.372558,"scheduling_latency_s":1.37258}1473vm-test-run-nixbot-gitlab> gitlab # [  163.923837] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.385Z","message":"No existing merge request to be cleaned up."}1474vm-test-run-nixbot-gitlab> gitlab # [  163.925310] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.391Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8ab77bedec9433f002f68850","created_at":"2026-07-04T12:38:22.980Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bde3c85a64974027","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"5981b5ecb35efb52d458addf2f9665c5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.002Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8ab77bedec9433f002f68850: done: 0.016116 sec","job_status":"done","queue_duration_s":1.372558,"scheduling_latency_s":1.37258,"redis_calls":10,"redis_duration_s":0.002274,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000922,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001352,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010327,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016116,"completed_at":"2026-07-04T12:38:24.391Z","load_balancing_strategy":"primary","db_duration_s":0.001742,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1475vm-test-run-nixbot-gitlab> gitlab # [  163.927719] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.392Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"eb6becee3406209149d34d47","created_at":"2026-07-04T12:38:22.980Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bde3c85a64974027","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"5981b5ecb35efb52d458addf2f9665c5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.002Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-eb6becee3406209149d34d47: start","job_status":"start","queue_duration_s":1.389937,"scheduling_latency_s":1.389959}1476vm-test-run-nixbot-gitlab> gitlab # [  163.939712] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.402Z","message":"No existing merge request to be cleaned up."}1477vm-test-run-nixbot-gitlab> gitlab # [  163.943115] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.408Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"eb6becee3406209149d34d47","created_at":"2026-07-04T12:38:22.980Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bde3c85a64974027","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"5981b5ecb35efb52d458addf2f9665c5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.002Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-eb6becee3406209149d34d47: done: 0.016155 sec","job_status":"done","queue_duration_s":1.389937,"scheduling_latency_s":1.389959,"redis_calls":10,"redis_duration_s":0.002245,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000864,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001381,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010379,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016155,"completed_at":"2026-07-04T12:38:24.408Z","load_balancing_strategy":"primary","db_duration_s":0.001458,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1478vm-test-run-nixbot-gitlab> gitlab # [  164.064361] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.409Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"18d0fe9fa155245a3cf63cf6","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-18d0fe9fa155245a3cf63cf6: start","job_status":"start","queue_duration_s":1.249596,"scheduling_latency_s":1.249617}1479vm-test-run-nixbot-gitlab> gitlab # [  164.065986] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.431Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"18d0fe9fa155245a3cf63cf6","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-18d0fe9fa155245a3cf63cf6: done: 0.021257 sec","job_status":"done","queue_duration_s":1.249596,"scheduling_latency_s":1.249617,"redis_calls":16,"redis_duration_s":0.003965,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000438,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001089,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002438,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013111,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000352,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021257,"completed_at":"2026-07-04T12:38:24.431Z","load_balancing_strategy":"primary","db_duration_s":0.000422,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1480vm-test-run-nixbot-gitlab> gitlab # [  164.086859] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.432Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0dc927272cbe7ea3e9229a38","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0dc927272cbe7ea3e9229a38: start","job_status":"start","queue_duration_s":1.272137,"scheduling_latency_s":1.272157}1481vm-test-run-nixbot-gitlab> gitlab # [  164.106842] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.456Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0dc927272cbe7ea3e9229a38","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0dc927272cbe7ea3e9229a38: done: 0.023758 sec","job_status":"done","queue_duration_s":1.272137,"scheduling_latency_s":1.272157,"redis_calls":13,"redis_duration_s":0.010344,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00203,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.008314,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010674,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023758,"completed_at":"2026-07-04T12:38:24.456Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1482vm-test-run-nixbot-gitlab> gitlab # [  164.198314] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.457Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"01f6c495a5ea7f54dd1ce595","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-01f6c495a5ea7f54dd1ce595: start","job_status":"start","queue_duration_s":1.297186,"scheduling_latency_s":1.297208}1483vm-test-run-nixbot-gitlab> gitlab # [  164.199945] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.476Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"01f6c495a5ea7f54dd1ce595","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-01f6c495a5ea7f54dd1ce595: done: 0.019323 sec","job_status":"done","queue_duration_s":1.297186,"scheduling_latency_s":1.297208,"redis_calls":13,"redis_duration_s":0.006741,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001122,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005619,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010557,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019323,"completed_at":"2026-07-04T12:38:24.476Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1484vm-test-run-nixbot-gitlab> gitlab # [  164.250191] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.478Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6204920458ab5522f7bd32c8","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6204920458ab5522f7bd32c8: start","job_status":"start","queue_duration_s":1.317954,"scheduling_latency_s":1.317976}1485vm-test-run-nixbot-gitlab> gitlab # [  164.270189] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.493Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6204920458ab5522f7bd32c8","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6204920458ab5522f7bd32c8: done: 0.015548 sec","job_status":"done","queue_duration_s":1.317954,"scheduling_latency_s":1.317976,"redis_calls":13,"redis_duration_s":0.002808,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000898,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00191,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01041,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015548,"completed_at":"2026-07-04T12:38:24.493Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1486vm-test-run-nixbot-gitlab> gitlab # [  164.353184] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.495Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34a742201ca771f786378b98","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34a742201ca771f786378b98: start","job_status":"start","queue_duration_s":1.334847,"scheduling_latency_s":1.334869}1487vm-test-run-nixbot-gitlab> gitlab # [  164.354791] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.510Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34a742201ca771f786378b98","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34a742201ca771f786378b98: done: 0.015387 sec","job_status":"done","queue_duration_s":1.334847,"scheduling_latency_s":1.334869,"redis_calls":13,"redis_duration_s":0.0030470000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000938,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002109,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010351,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015387,"completed_at":"2026-07-04T12:38:24.510Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1488vm-test-run-nixbot-gitlab> gitlab # [  164.468199] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.511Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"09c134199e73ff71d0e17321","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-09c134199e73ff71d0e17321: start","job_status":"start","queue_duration_s":1.351554,"scheduling_latency_s":1.351575}1489vm-test-run-nixbot-gitlab> gitlab # [  164.469842] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.534Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"09c134199e73ff71d0e17321","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-09c134199e73ff71d0e17321: done: 0.02224 sec","job_status":"done","queue_duration_s":1.351554,"scheduling_latency_s":1.351575,"redis_calls":13,"redis_duration_s":0.007432,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001034,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006398,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012033,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02224,"completed_at":"2026-07-04T12:38:24.533Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1490vm-test-run-nixbot-gitlab> gitlab # [  164.495217] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.535Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a709ed43f02ceabffe4b4aa9","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a709ed43f02ceabffe4b4aa9: start","job_status":"start","queue_duration_s":1.375404,"scheduling_latency_s":1.375433}1491vm-test-run-nixbot-gitlab> gitlab # [  164.575108] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.552Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a709ed43f02ceabffe4b4aa9","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a709ed43f02ceabffe4b4aa9: done: 0.016809 sec","job_status":"done","queue_duration_s":1.375404,"scheduling_latency_s":1.375433,"redis_calls":13,"redis_duration_s":0.0029950000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000912,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002083,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010236,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016809,"completed_at":"2026-07-04T12:38:24.552Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1492vm-test-run-nixbot-gitlab> gitlab # [  164.576812] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.553Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"624f867864c734a33f7699e8","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-624f867864c734a33f7699e8: start","job_status":"start","queue_duration_s":1.393493,"scheduling_latency_s":1.393556}1493vm-test-run-nixbot-gitlab> gitlab # [  164.595776] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.569Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"624f867864c734a33f7699e8","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-624f867864c734a33f7699e8: done: 0.015576 sec","job_status":"done","queue_duration_s":1.393493,"scheduling_latency_s":1.393556,"redis_calls":13,"redis_duration_s":0.003083,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000989,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002094,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009869,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015576,"completed_at":"2026-07-04T12:38:24.569Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1494vm-test-run-nixbot-gitlab> gitlab # [  164.665320] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.570Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cbe544080beba37e7f62c52f","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cbe544080beba37e7f62c52f: start","job_status":"start","queue_duration_s":1.410593,"scheduling_latency_s":1.410614}1495vm-test-run-nixbot-gitlab> gitlab # [  164.676225] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.586Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cbe544080beba37e7f62c52f","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cbe544080beba37e7f62c52f: done: 0.015941 sec","job_status":"done","queue_duration_s":1.410593,"scheduling_latency_s":1.410614,"redis_calls":13,"redis_duration_s":0.0033369999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001064,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002273,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009977,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015941,"completed_at":"2026-07-04T12:38:24.586Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1496vm-test-run-nixbot-gitlab> gitlab # [  164.780820] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.588Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"70afb1b34227355bcb90120c","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-70afb1b34227355bcb90120c: start","job_status":"start","queue_duration_s":1.428002,"scheduling_latency_s":1.428024}1497vm-test-run-nixbot-gitlab> gitlab # [  164.782297] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.604Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"70afb1b34227355bcb90120c","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-70afb1b34227355bcb90120c: done: 0.016098 sec","job_status":"done","queue_duration_s":1.428002,"scheduling_latency_s":1.428024,"redis_calls":13,"redis_duration_s":0.002993,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000934,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002059,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010038,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016098,"completed_at":"2026-07-04T12:38:24.604Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1498vm-test-run-nixbot-gitlab> gitlab # [  164.820274] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.605Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"40b3ffbc1c5af7c255258fd4","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-40b3ffbc1c5af7c255258fd4: start","job_status":"start","queue_duration_s":1.445568,"scheduling_latency_s":1.44559}1499vm-test-run-nixbot-gitlab> gitlab # [  164.840298] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.621Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"40b3ffbc1c5af7c255258fd4","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-40b3ffbc1c5af7c255258fd4: done: 0.01569 sec","job_status":"done","queue_duration_s":1.445568,"scheduling_latency_s":1.44559,"redis_calls":13,"redis_duration_s":0.00316,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001042,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002118,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010019,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01569,"completed_at":"2026-07-04T12:38:24.621Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1500vm-test-run-nixbot-gitlab> gitlab # [  164.928678] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.623Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b6ee7a87050a612519bb479","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b6ee7a87050a612519bb479: start","job_status":"start","queue_duration_s":1.462991,"scheduling_latency_s":1.463013}1501vm-test-run-nixbot-gitlab> gitlab # [  164.930110] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.639Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b6ee7a87050a612519bb479","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b6ee7a87050a612519bb479: done: 0.015868 sec","job_status":"done","queue_duration_s":1.462991,"scheduling_latency_s":1.463013,"redis_calls":13,"redis_duration_s":0.003304,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000974,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00233,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010056,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015868,"completed_at":"2026-07-04T12:38:24.639Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1502vm-test-run-nixbot-gitlab> gitlab # [  164.984181] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.640Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e4ba3603765591b21bc0841d","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e4ba3603765591b21bc0841d: start","job_status":"start","queue_duration_s":1.480295,"scheduling_latency_s":1.480317}1503vm-test-run-nixbot-gitlab> gitlab # [  165.004207] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.660Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e4ba3603765591b21bc0841d","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e4ba3603765591b21bc0841d: done: 0.020028 sec","job_status":"done","queue_duration_s":1.480295,"scheduling_latency_s":1.480317,"redis_calls":13,"redis_duration_s":0.002947,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000901,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002046,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01014,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020028,"completed_at":"2026-07-04T12:38:24.660Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1504vm-test-run-nixbot-gitlab> gitlab # [  165.067448] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.662Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f8cf59573ffedd28a7ea0550","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f8cf59573ffedd28a7ea0550: start","job_status":"start","queue_duration_s":1.502158,"scheduling_latency_s":1.502179}1505vm-test-run-nixbot-gitlab> gitlab # [  165.087516] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.678Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f8cf59573ffedd28a7ea0550","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f8cf59573ffedd28a7ea0550: done: 0.016226 sec","job_status":"done","queue_duration_s":1.502158,"scheduling_latency_s":1.502179,"redis_calls":13,"redis_duration_s":0.0029900000000000005,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000943,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002047,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010194,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016226,"completed_at":"2026-07-04T12:38:24.678Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1506vm-test-run-nixbot-gitlab> gitlab # [  165.151184] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.680Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68aa1c83cfc473f2f8495b57","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68aa1c83cfc473f2f8495b57: start","job_status":"start","queue_duration_s":1.51976,"scheduling_latency_s":1.519781}1507vm-test-run-nixbot-gitlab> gitlab # [  165.171354] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.695Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68aa1c83cfc473f2f8495b57","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68aa1c83cfc473f2f8495b57: done: 0.015642 sec","job_status":"done","queue_duration_s":1.51976,"scheduling_latency_s":1.519781,"redis_calls":13,"redis_duration_s":0.002998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001016,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001982,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009959,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015642,"completed_at":"2026-07-04T12:38:24.695Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1508vm-test-run-nixbot-gitlab> gitlab # [  165.234215] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.697Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"edb07f8e250323f6f2a8f4f4","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-edb07f8e250323f6f2a8f4f4: start","job_status":"start","queue_duration_s":1.536781,"scheduling_latency_s":1.536802}1509vm-test-run-nixbot-gitlab> gitlab # [  165.254184] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.712Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"edb07f8e250323f6f2a8f4f4","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-edb07f8e250323f6f2a8f4f4: done: 0.015828 sec","job_status":"done","queue_duration_s":1.536781,"scheduling_latency_s":1.536802,"redis_calls":13,"redis_duration_s":0.0030090000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000933,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002076,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009872,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015828,"completed_at":"2026-07-04T12:38:24.712Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1510vm-test-run-nixbot-gitlab> gitlab # [  165.365484] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.714Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4265c679ac13098b58da06ec","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4265c679ac13098b58da06ec: start","job_status":"start","queue_duration_s":1.554145,"scheduling_latency_s":1.554167}1511vm-test-run-nixbot-gitlab> gitlab # [  165.366958] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.733Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4265c679ac13098b58da06ec","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4265c679ac13098b58da06ec: done: 0.0192 sec","job_status":"done","queue_duration_s":1.554145,"scheduling_latency_s":1.554167,"redis_calls":13,"redis_duration_s":0.0037080000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001083,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002625,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012303,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0192,"completed_at":"2026-07-04T12:38:24.733Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1512vm-test-run-nixbot-gitlab> gitlab # [  165.398820] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.735Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dbbb4db98a0ff6edce263d26","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dbbb4db98a0ff6edce263d26: start","job_status":"start","queue_duration_s":1.574929,"scheduling_latency_s":1.574957}1513vm-test-run-nixbot-gitlab> gitlab # [  165.418899] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.752Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dbbb4db98a0ff6edce263d26","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dbbb4db98a0ff6edce263d26: done: 0.017163 sec","job_status":"done","queue_duration_s":1.574929,"scheduling_latency_s":1.574957,"redis_calls":13,"redis_duration_s":0.003195,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001046,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002149,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010557,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017163,"completed_at":"2026-07-04T12:38:24.752Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1514vm-test-run-nixbot-gitlab> gitlab # [  165.510451] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.754Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"39591f942a00696017bb24c3","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-39591f942a00696017bb24c3: start","job_status":"start","queue_duration_s":1.593768,"scheduling_latency_s":1.59379}1515vm-test-run-nixbot-gitlab> gitlab # [  165.511754] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.770Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"39591f942a00696017bb24c3","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-39591f942a00696017bb24c3: done: 0.015965 sec","job_status":"done","queue_duration_s":1.593768,"scheduling_latency_s":1.59379,"redis_calls":13,"redis_duration_s":0.0029790000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000896,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002083,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00995,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015965,"completed_at":"2026-07-04T12:38:24.769Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1516vm-test-run-nixbot-gitlab> gitlab # [  165.608975] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.771Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1df812d61f3ea98bf4d19af4","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1df812d61f3ea98bf4d19af4: start","job_status":"start","queue_duration_s":1.611266,"scheduling_latency_s":1.611287}1517vm-test-run-nixbot-gitlab> gitlab # [  165.609818] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.787Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1df812d61f3ea98bf4d19af4","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1df812d61f3ea98bf4d19af4: done: 0.016069 sec","job_status":"done","queue_duration_s":1.611266,"scheduling_latency_s":1.611287,"redis_calls":13,"redis_duration_s":0.00305,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000886,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002164,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010047,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016069,"completed_at":"2026-07-04T12:38:24.787Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1518vm-test-run-nixbot-gitlab> gitlab # [  165.643845] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.789Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"caca8ef255bf54366b4e059b","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-caca8ef255bf54366b4e059b: start","job_status":"start","queue_duration_s":1.628801,"scheduling_latency_s":1.628823}1519vm-test-run-nixbot-gitlab> gitlab # [  165.663944] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.804Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"caca8ef255bf54366b4e059b","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-caca8ef255bf54366b4e059b: done: 0.015624 sec","job_status":"done","queue_duration_s":1.628801,"scheduling_latency_s":1.628823,"redis_calls":13,"redis_duration_s":0.003101,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00097,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002131,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010108,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015624,"completed_at":"2026-07-04T12:38:24.804Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1520vm-test-run-nixbot-gitlab> gitlab # [  165.726989] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.806Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9a7a8a54d67c0d988d8a23fc","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9a7a8a54d67c0d988d8a23fc: start","job_status":"start","queue_duration_s":1.645959,"scheduling_latency_s":1.64598}1521vm-test-run-nixbot-gitlab> gitlab # [  165.798917] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.821Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9a7a8a54d67c0d988d8a23fc","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9a7a8a54d67c0d988d8a23fc: done: 0.015539 sec","job_status":"done","queue_duration_s":1.645959,"scheduling_latency_s":1.64598,"redis_calls":13,"redis_duration_s":0.0031219999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001102,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00202,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009971,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015539,"completed_at":"2026-07-04T12:38:24.821Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1522vm-test-run-nixbot-gitlab> gitlab # [  165.807963] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.823Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b90760a049f4ed0164ece7d","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b90760a049f4ed0164ece7d: start","job_status":"start","queue_duration_s":1.663028,"scheduling_latency_s":1.663049}1523vm-test-run-nixbot-gitlab> gitlab # [  165.827997] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.839Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b90760a049f4ed0164ece7d","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b90760a049f4ed0164ece7d: done: 0.016172 sec","job_status":"done","queue_duration_s":1.663028,"scheduling_latency_s":1.663049,"redis_calls":13,"redis_duration_s":0.0030210000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001015,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002006,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010074,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016172,"completed_at":"2026-07-04T12:38:24.839Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1524vm-test-run-nixbot-gitlab> gitlab # [  165.926280] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.841Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"23927c356504667f9d97e63c","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-23927c356504667f9d97e63c: start","job_status":"start","queue_duration_s":1.680776,"scheduling_latency_s":1.680799}1525vm-test-run-nixbot-gitlab> gitlab # [  165.927463] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.856Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"23927c356504667f9d97e63c","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-23927c356504667f9d97e63c: done: 0.015827 sec","job_status":"done","queue_duration_s":1.680776,"scheduling_latency_s":1.680799,"redis_calls":13,"redis_duration_s":0.0030819999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000985,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002097,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010167,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015827,"completed_at":"2026-07-04T12:38:24.856Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1526vm-test-run-nixbot-gitlab> gitlab # [  166.006601] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.858Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33f262de15f93bbe0fd7a9d5","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33f262de15f93bbe0fd7a9d5: start","job_status":"start","queue_duration_s":1.698046,"scheduling_latency_s":1.698068}1527vm-test-run-nixbot-gitlab> gitlab # [  166.007797] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.875Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33f262de15f93bbe0fd7a9d5","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33f262de15f93bbe0fd7a9d5: done: 0.016745 sec","job_status":"done","queue_duration_s":1.698046,"scheduling_latency_s":1.698068,"redis_calls":13,"redis_duration_s":0.004091,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002111,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00198,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010524,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016745,"completed_at":"2026-07-04T12:38:24.875Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1528vm-test-run-nixbot-gitlab> gitlab # [  166.087193] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.879Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fe90dfc15f7efe86009245df","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fe90dfc15f7efe86009245df: start","job_status":"start","queue_duration_s":1.719662,"scheduling_latency_s":1.719684}1529vm-test-run-nixbot-gitlab> gitlab # [  166.088853] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.905Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fe90dfc15f7efe86009245df","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fe90dfc15f7efe86009245df: done: 0.025481 sec","job_status":"done","queue_duration_s":1.719662,"scheduling_latency_s":1.719684,"redis_calls":13,"redis_duration_s":0.006283,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00125,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005033,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01293,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025481,"completed_at":"2026-07-04T12:38:24.905Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1530vm-test-run-nixbot-gitlab> gitlab # [  166.134970] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.907Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"66f1208c121dc1e417bea68c","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-66f1208c121dc1e417bea68c: start","job_status":"start","queue_duration_s":1.746695,"scheduling_latency_s":1.746724}1531vm-test-run-nixbot-gitlab> gitlab # [  166.214699] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.923Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"66f1208c121dc1e417bea68c","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-66f1208c121dc1e417bea68c: done: 0.016782 sec","job_status":"done","queue_duration_s":1.746695,"scheduling_latency_s":1.746724,"redis_calls":13,"redis_duration_s":0.003124,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000908,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002216,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01033,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016782,"completed_at":"2026-07-04T12:38:24.923Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1532vm-test-run-nixbot-gitlab> gitlab # [  166.216701] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.925Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e197250ee789b00b397cd86d","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e197250ee789b00b397cd86d: start","job_status":"start","queue_duration_s":1.765124,"scheduling_latency_s":1.765145}1533vm-test-run-nixbot-gitlab> gitlab # [  166.235194] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.941Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e197250ee789b00b397cd86d","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e197250ee789b00b397cd86d: done: 0.016062 sec","job_status":"done","queue_duration_s":1.765124,"scheduling_latency_s":1.765145,"redis_calls":13,"redis_duration_s":0.003117,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000877,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00224,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00999,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016062,"completed_at":"2026-07-04T12:38:24.941Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1534vm-test-run-nixbot-gitlab> gitlab # [  166.328405] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.942Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"29c3a51f72ae1778c613ed5f","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-29c3a51f72ae1778c613ed5f: start","job_status":"start","queue_duration_s":1.782543,"scheduling_latency_s":1.782565}1535vm-test-run-nixbot-gitlab> gitlab # [  166.330052] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.958Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"29c3a51f72ae1778c613ed5f","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-29c3a51f72ae1778c613ed5f: done: 0.016224 sec","job_status":"done","queue_duration_s":1.782543,"scheduling_latency_s":1.782565,"redis_calls":13,"redis_duration_s":0.002974,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000884,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00209,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010132,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016224,"completed_at":"2026-07-04T12:38:24.958Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1536vm-test-run-nixbot-gitlab> gitlab # [  166.417361] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:24.960Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"05ca5c654fafea2bad88bd96","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-05ca5c654fafea2bad88bd96: start","job_status":"start","queue_duration_s":1.80026,"scheduling_latency_s":1.800281}1537vm-test-run-nixbot-gitlab> gitlab # [  166.729014] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 2190415867 wd_nsec: 21904146011538vm-test-run-nixbot-gitlab> gitlab # [  166.487720] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:27.438Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"05ca5c654fafea2bad88bd96","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-05ca5c654fafea2bad88bd96: done: 2.478231 sec","job_status":"done","queue_duration_s":1.80026,"scheduling_latency_s":1.800281,"redis_calls":13,"redis_duration_s":0.003979,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001818,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002161,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01156,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":2.478231,"completed_at":"2026-07-04T12:38:27.438Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1539vm-test-run-nixbot-gitlab> gitlab # [  166.550566] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:27.501Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc9b126151ff078b31eeaa13","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc9b126151ff078b31eeaa13: start","job_status":"start","queue_duration_s":4.341647,"scheduling_latency_s":4.341686}1540vm-test-run-nixbot-gitlab> gitlab # [  166.586691] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:27.538Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc9b126151ff078b31eeaa13","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc9b126151ff078b31eeaa13: done: 0.036233 sec","job_status":"done","queue_duration_s":4.341647,"scheduling_latency_s":4.341686,"redis_calls":13,"redis_duration_s":0.003107,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000917,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00219,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01154,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.036233,"completed_at":"2026-07-04T12:38:27.537Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1541vm-test-run-nixbot-gitlab> gitlab # [  166.881372] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:27.831Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2707f83ddf129804159a8909","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2707f83ddf129804159a8909: start","job_status":"start","queue_duration_s":4.671371,"scheduling_latency_s":4.67142}1542vm-test-run-nixbot-gitlab> gitlab # [  166.971380] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:27.922Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2707f83ddf129804159a8909","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2707f83ddf129804159a8909: done: 0.090552 sec","job_status":"done","queue_duration_s":4.671371,"scheduling_latency_s":4.67142,"redis_calls":13,"redis_duration_s":0.008672,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002595,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006077,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028078,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.090552,"completed_at":"2026-07-04T12:38:27.922Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1543vm-test-run-nixbot-gitlab> gitlab # [  167.062778] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:28.012Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a141b00763c5bf99107f32a2","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a141b00763c5bf99107f32a2: start","job_status":"start","queue_duration_s":4.852405,"scheduling_latency_s":4.852476}1544vm-test-run-nixbot-gitlab> gitlab # [  167.104387] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:28.055Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a141b00763c5bf99107f32a2","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a141b00763c5bf99107f32a2: done: 0.042596 sec","job_status":"done","queue_duration_s":4.852405,"scheduling_latency_s":4.852476,"redis_calls":13,"redis_duration_s":0.007629,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002283,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005346,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026089,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.042596,"completed_at":"2026-07-04T12:38:28.055Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1545vm-test-run-nixbot-gitlab> gitlab # [  167.170468] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:28.120Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"290aa0d058a0038dfe7157d5","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-290aa0d058a0038dfe7157d5: start","job_status":"start","queue_duration_s":4.960166,"scheduling_latency_s":4.960234}1546vm-test-run-nixbot-gitlab> gitlab # [  167.226276] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:28.177Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"290aa0d058a0038dfe7157d5","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-290aa0d058a0038dfe7157d5: done: 0.056797 sec","job_status":"done","queue_duration_s":4.960166,"scheduling_latency_s":4.960234,"redis_calls":13,"redis_duration_s":0.00514,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001713,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003427,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025533,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.056797,"completed_at":"2026-07-04T12:38:28.177Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1547vm-test-run-nixbot-gitlab> gitlab # [  167.291114] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:28.242Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b770ba11db11d536e1ff4e3d","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b770ba11db11d536e1ff4e3d: start","job_status":"start","queue_duration_s":5.081634,"scheduling_latency_s":5.081705}1548vm-test-run-nixbot-gitlab> gitlab # [  167.347265] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:28.298Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b770ba11db11d536e1ff4e3d","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b770ba11db11d536e1ff4e3d: done: 0.056469 sec","job_status":"done","queue_duration_s":5.081634,"scheduling_latency_s":5.081705,"redis_calls":13,"redis_duration_s":0.0049499999999999995,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001553,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003397,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026427,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.056469,"completed_at":"2026-07-04T12:38:28.298Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1549vm-test-run-nixbot-gitlab> gitlab # [  167.413114] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:28.364Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb6643a798e6679b627bffdb","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb6643a798e6679b627bffdb: start","job_status":"start","queue_duration_s":5.203612,"scheduling_latency_s":5.203687}1550vm-test-run-nixbot-gitlab> gitlab # [  167.456547] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:28.407Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb6643a798e6679b627bffdb","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb6643a798e6679b627bffdb: done: 0.043528 sec","job_status":"done","queue_duration_s":5.203612,"scheduling_latency_s":5.203687,"redis_calls":13,"redis_duration_s":0.008109,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002703,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005406,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026757,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.043528,"completed_at":"2026-07-04T12:38:28.407Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1551vm-test-run-nixbot-gitlab> gitlab # [  167.462226] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:28.413Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"89422f4fa973ab16ed7d1a60","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-89422f4fa973ab16ed7d1a60: start","job_status":"start","queue_duration_s":5.253023,"scheduling_latency_s":5.253075}1552vm-test-run-nixbot-gitlab> gitlab # [  167.499558] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:28.450Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"89422f4fa973ab16ed7d1a60","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-89422f4fa973ab16ed7d1a60: done: 0.037275 sec","job_status":"done","queue_duration_s":5.253023,"scheduling_latency_s":5.253075,"redis_calls":13,"redis_duration_s":0.00733,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00242,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00491,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021343,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.037275,"completed_at":"2026-07-04T12:38:28.450Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1553vm-test-run-nixbot-gitlab> gitlab # [  167.532497] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:28.457Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5ff0e953887325b981189917","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5ff0e953887325b981189917: start","job_status":"start","queue_duration_s":5.297183,"scheduling_latency_s":5.297227}1554vm-test-run-nixbot-gitlab> gitlab # [  167.552290] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:28.485Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5ff0e953887325b981189917","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5ff0e953887325b981189917: done: 0.028075 sec","job_status":"done","queue_duration_s":5.297183,"scheduling_latency_s":5.297227,"redis_calls":13,"redis_duration_s":0.011247,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002948,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.008299,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011698,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028075,"completed_at":"2026-07-04T12:38:28.485Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1555vm-test-run-nixbot-gitlab> gitlab # [  167.673380] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:28.488Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e2c8a3c660a84b95b7ba6c74","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e2c8a3c660a84b95b7ba6c74: start","job_status":"start","queue_duration_s":5.327997,"scheduling_latency_s":5.32802}1556vm-test-run-nixbot-gitlab> gitlab # [  167.675774] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:28.511Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e2c8a3c660a84b95b7ba6c74","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e2c8a3c660a84b95b7ba6c74: done: 0.022941 sec","job_status":"done","queue_duration_s":5.327997,"scheduling_latency_s":5.32802,"redis_calls":13,"redis_duration_s":0.010017999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003021,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006997,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010528,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022941,"completed_at":"2026-07-04T12:38:28.511Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1557vm-test-run-nixbot-gitlab> gitlab # [  167.697188] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:28.512Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c082126505a21fdb40abbc4d","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c082126505a21fdb40abbc4d: start","job_status":"start","queue_duration_s":5.352258,"scheduling_latency_s":5.352281}1558vm-test-run-nixbot-gitlab> gitlab # [  167.717400] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:28.528Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c082126505a21fdb40abbc4d","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c082126505a21fdb40abbc4d: done: 0.015664 sec","job_status":"done","queue_duration_s":5.352258,"scheduling_latency_s":5.352281,"redis_calls":13,"redis_duration_s":0.003299,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000943,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002356,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009855,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015664,"completed_at":"2026-07-04T12:38:28.528Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1559vm-test-run-nixbot-gitlab> gitlab # [  167.818204] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:28.529Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9703c7fb8984f625bfbb389d","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9703c7fb8984f625bfbb389d: start","job_status":"start","queue_duration_s":5.369211,"scheduling_latency_s":5.369232}1560vm-test-run-nixbot-gitlab> gitlab # [  167.819825] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:28.545Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9703c7fb8984f625bfbb389d","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9703c7fb8984f625bfbb389d: done: 0.015771 sec","job_status":"done","queue_duration_s":5.369211,"scheduling_latency_s":5.369232,"redis_calls":13,"redis_duration_s":0.0030670000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000992,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002075,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009447,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015771,"completed_at":"2026-07-04T12:38:28.545Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1561vm-test-run-nixbot-gitlab> gitlab # [  167.923255] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:28.547Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d53ec204606bb1aaa070026","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d53ec204606bb1aaa070026: start","job_status":"start","queue_duration_s":5.387006,"scheduling_latency_s":5.387027}1562vm-test-run-nixbot-gitlab> gitlab # [  167.925259] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:28.564Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d53ec204606bb1aaa070026","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d53ec204606bb1aaa070026: done: 0.017235 sec","job_status":"done","queue_duration_s":5.387006,"scheduling_latency_s":5.387027,"redis_calls":13,"redis_duration_s":0.002947,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000934,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002013,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010187,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017235,"completed_at":"2026-07-04T12:38:28.564Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1563vm-test-run-nixbot-gitlab> gitlab # [  167.943272] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:28.565Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d36e3432921342aa6e2e2f10","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d36e3432921342aa6e2e2f10: start","job_status":"start","queue_duration_s":5.405573,"scheduling_latency_s":5.405594}1564vm-test-run-nixbot-gitlab> gitlab # [  167.963171] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:28.581Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d36e3432921342aa6e2e2f10","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d36e3432921342aa6e2e2f10: done: 0.015813 sec","job_status":"done","queue_duration_s":5.405573,"scheduling_latency_s":5.405594,"redis_calls":13,"redis_duration_s":0.003633,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000984,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002649,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010257,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015813,"completed_at":"2026-07-04T12:38:28.581Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1565vm-test-run-nixbot-gitlab> gitlab # [  168.026333] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:28.583Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2814c37e762712d17f63bdc1","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2814c37e762712d17f63bdc1: start","job_status":"start","queue_duration_s":5.422772,"scheduling_latency_s":5.422793}1566vm-test-run-nixbot-gitlab> gitlab # [  168.108517] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:28.599Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2814c37e762712d17f63bdc1","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2814c37e762712d17f63bdc1: done: 0.016916 sec","job_status":"done","queue_duration_s":5.422772,"scheduling_latency_s":5.422793,"redis_calls":13,"redis_duration_s":0.004097,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001252,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002845,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010639,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016916,"completed_at":"2026-07-04T12:38:28.599Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1567vm-test-run-nixbot-gitlab> gitlab # [  168.110921] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:28.601Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d0bac82c9391f0b5245bbe15","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d0bac82c9391f0b5245bbe15: start","job_status":"start","queue_duration_s":5.441353,"scheduling_latency_s":5.441374}1568vm-test-run-nixbot-gitlab> gitlab # [  168.126959] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:28.618Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d0bac82c9391f0b5245bbe15","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d0bac82c9391f0b5245bbe15: done: 0.017283 sec","job_status":"done","queue_duration_s":5.441353,"scheduling_latency_s":5.441374,"redis_calls":13,"redis_duration_s":0.0043230000000000005,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001349,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002974,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009976,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017283,"completed_at":"2026-07-04T12:38:28.618Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1569vm-test-run-nixbot-gitlab> gitlab # [  168.217480] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:28.621Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c0bb2ef5259e3ea71d9eb19b","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c0bb2ef5259e3ea71d9eb19b: start","job_status":"start","queue_duration_s":5.460793,"scheduling_latency_s":5.460816}1570vm-test-run-nixbot-gitlab> gitlab # [  168.219261] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:28.642Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c0bb2ef5259e3ea71d9eb19b","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c0bb2ef5259e3ea71d9eb19b: done: 0.021326 sec","job_status":"done","queue_duration_s":5.460793,"scheduling_latency_s":5.460816,"redis_calls":13,"redis_duration_s":0.008527,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001684,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006843,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010585,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021326,"completed_at":"2026-07-04T12:38:28.642Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1571vm-test-run-nixbot-gitlab> gitlab # [  168.272131] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:28.643Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0703918baef1b7ce5b09161d","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0703918baef1b7ce5b09161d: start","job_status":"start","queue_duration_s":5.483451,"scheduling_latency_s":5.483473}1572vm-test-run-nixbot-gitlab> gitlab # [  168.356731] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:28.659Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0703918baef1b7ce5b09161d","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0703918baef1b7ce5b09161d: done: 0.015597 sec","job_status":"done","queue_duration_s":5.483451,"scheduling_latency_s":5.483473,"redis_calls":13,"redis_duration_s":0.0030700000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000899,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002171,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009937,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015597,"completed_at":"2026-07-04T12:38:28.659Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1573vm-test-run-nixbot-gitlab> gitlab # [  168.358911] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:28.660Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d3738f5067b64246d3f70a48","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d3738f5067b64246d3f70a48: start","job_status":"start","queue_duration_s":5.500357,"scheduling_latency_s":5.500378}1574vm-test-run-nixbot-gitlab> gitlab # [  168.373560] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:28.676Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d3738f5067b64246d3f70a48","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d3738f5067b64246d3f70a48: done: 0.016165 sec","job_status":"done","queue_duration_s":5.500357,"scheduling_latency_s":5.500378,"redis_calls":13,"redis_duration_s":0.0030570000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000893,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002164,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010173,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016165,"completed_at":"2026-07-04T12:38:28.676Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1575vm-test-run-nixbot-gitlab> gitlab # [  168.499112] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:28.678Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"504bb87d6d5b0a7b9e76291f","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-504bb87d6d5b0a7b9e76291f: start","job_status":"start","queue_duration_s":5.517843,"scheduling_latency_s":5.517865}1576vm-test-run-nixbot-gitlab> gitlab # [  168.500424] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:28.693Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"504bb87d6d5b0a7b9e76291f","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-504bb87d6d5b0a7b9e76291f: done: 0.015545 sec","job_status":"done","queue_duration_s":5.517843,"scheduling_latency_s":5.517865,"redis_calls":13,"redis_duration_s":0.002882,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00094,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001942,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00952,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015545,"completed_at":"2026-07-04T12:38:28.693Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1577vm-test-run-nixbot-gitlab> gitlab # [  168.518120] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:28.695Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84c71dc5aeb7d84c45811573","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84c71dc5aeb7d84c45811573: start","job_status":"start","queue_duration_s":5.535116,"scheduling_latency_s":5.535138}1578vm-test-run-nixbot-gitlab> gitlab # [  168.538110] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:38:28.710Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84c71dc5aeb7d84c45811573","created_at":"2026-07-04T12:38:23.045Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-bea508cd15714c47","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"278b55a550a8e3be41952519e172e1f2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:38:23.160Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84c71dc5aeb7d84c45811573: done: 0.01559 sec","job_status":"done","queue_duration_s":5.535116,"scheduling_latency_s":5.535138,"redis_calls":13,"redis_duration_s":0.003083,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000977,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002106,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009956,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01559,"completed_at":"2026-07-04T12:38:28.710Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1579vm-test-run-nixbot-gitlab> gitlab # [  175.432915] setup-gitlab-start[2744]: W, [2026-07-04T12:38:36.383500 #2744]  WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1580vm-test-run-nixbot-gitlab> gitlab # [  199.940470] runuser[2743]: pam_unix(runuser:session): session closed for user gitlab1581vm-test-run-nixbot-gitlab> gitlab # [  199.947145] setup-gitlab-start[2972]: ++ jq -r .token1582vm-test-run-nixbot-gitlab> gitlab # [  199.949431] setup-gitlab-start[2973]: +++ date -d '+7 days' +%F1583vm-test-run-nixbot-gitlab> gitlab # [  199.952308] setup-gitlab-start[2971]: ++ curl -fs -X POST -H 'Authorization: Bearer 7cabaaef6f9590979a7fef01ec7730d86857a6110808247748a0c12c4730a6ac' -H 'Content-Type: application/json' -d '{"name":"nixbot","scopes":["api"],"expires_at":"2026-07-11"}' http://gitlab/api/v4/users/1/personal_access_tokens1584vm-test-run-nixbot-gitlab> gitlab # [  200.150382] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:01.100Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"a2c8e5b697d43ffa4895cd32","created_at":"2026-07-04T12:39:01.074Z","trace_propagation_headers":{"sentry-trace":"fb2e26a1c2c1491ca1cd6a9a14feabf5-e08489fc81db45ca","baggage":"sentry-trace_id=fb2e26a1c2c1491ca1cd6a9a14feabf5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPJ7BHJMDWXGMDQWB8ERJJ1","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:08963546953dca552c18883018e9cd97e4b7e06fd1f97c2d449a9856196ebe14","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:01.096Z","job_size_bytes":451,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-a2c8e5b697d43ffa4895cd32: start","job_status":"start","queue_duration_s":0.004,"scheduling_latency_s":0.004045}1585vm-test-run-nixbot-gitlab> gitlab # [  200.215855] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:01 +0000] "POST /api/v4/users/1/personal_access_tokens HTTP/1.1" 201 267 "" "curl/8.20.0" 2481586vm-test-run-nixbot-gitlab> gitlab # [  200.221271] setup-gitlab-start[2696]: + TOKEN=glpat-chNOUPOfvDkXEMZZsZAOy286MQp1OjEH.01.0w02nn8681587vm-test-run-nixbot-gitlab> gitlab # [  200.222972] setup-gitlab-start[2696]: + test -n glpat-chNOUPOfvDkXEMZZsZAOy286MQp1OjEH.01.0w02nn8681588vm-test-run-nixbot-gitlab> gitlab # [  200.224165] setup-gitlab-start[2696]: + test glpat-chNOUPOfvDkXEMZZsZAOy286MQp1OjEH.01.0w02nn868 '!=' null1589vm-test-run-nixbot-gitlab> gitlab # [  200.225358] setup-gitlab-start[2696]: + mkdir -p /var/lib/secrets1590vm-test-run-nixbot-gitlab> gitlab # [  200.226571] setup-gitlab-start[2696]: + echo glpat-chNOUPOfvDkXEMZZsZAOy286MQp1OjEH.01.0w02nn8681591vm-test-run-nixbot-gitlab> gitlab # [  200.227868] setup-gitlab-start[2696]: + chmod 644 /var/lib/secrets/gitlab-token1592vm-test-run-nixbot-gitlab> gitlab # [  200.235802] setup-gitlab-start[2696]: + curl -fs -X POST -H 'PRIVATE-TOKEN: glpat-chNOUPOfvDkXEMZZsZAOy286MQp1OjEH.01.0w02nn868' -H 'Content-Type: application/json' -d '{"name":"test-flake","visibility":"public","topics":["build-with-buildbot"]}' http://gitlab/api/v4/projects1593vm-test-run-nixbot-gitlab> gitlab # [  200.353176] sidekiq-cluster[2150]: {"severity":"WARN","time":"2026-07-04T12:39:01.303Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"a2c8e5b697d43ffa4895cd32","created_at":"2026-07-04T12:39:01.074Z","trace_propagation_headers":{"sentry-trace":"fb2e26a1c2c1491ca1cd6a9a14feabf5-e08489fc81db45ca","baggage":"sentry-trace_id=fb2e26a1c2c1491ca1cd6a9a14feabf5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPJ7BHJMDWXGMDQWB8ERJJ1","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:08963546953dca552c18883018e9cd97e4b7e06fd1f97c2d449a9856196ebe14","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:01.096Z","job_size_bytes":451,"pid":2150,"sidekiq_tid":"lymu","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-a2c8e5b697d43ffa4895cd32: fail: 0.202449 sec","job_status":"fail","queue_duration_s":0.004,"scheduling_latency_s":0.004045,"redis_calls":11,"redis_duration_s":0.004578,"redis_read_bytes":137,"redis_write_bytes":1317,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001396,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002407,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000775,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.09836,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000601473,"duration_s":0.202449,"completed_at":"2026-07-04T12:39:01.302Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.08068,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1594vm-test-run-nixbot-gitlab> gitlab # [  200.937283] gitaly[1392]: time="2026-07-04T12:39:01.745Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7BTK51X1250TDH00879N error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-04T12:39:11.004" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T12:39:01.744" pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1595vm-test-run-nixbot-gitlab> gitlab # [  200.939492] gitaly[1392]: time="2026-07-04T12:39:01.745Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7BTK51X1250TDH00879N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-04T12:39:11.004" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T12:39:01.744" grpc.time_ms=0.379 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1596vm-test-run-nixbot-gitlab> gitlab # [  200.941335] gitaly[1392]: time="2026-07-04T12:39:01.887Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7BTK51X1250TDH00879N error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-04T12:39:11.146" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T12:39:01.887" pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1597vm-test-run-nixbot-gitlab> gitlab # [  200.942120] gitaly[1392]: time="2026-07-04T12:39:01.887Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7BTK51X1250TDH00879N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-04T12:39:11.146" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T12:39:01.887" grpc.time_ms=0.255 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1598vm-test-run-nixbot-gitlab> gitlab # [  201.012399] gitaly[1392]: time="2026-07-04T12:39:01.922Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7BTK51X1250TDH00879N diskcache=d57d0e9e-103f-4ba0-b991-ab169e0caf33 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-04T12:39:11.155" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T12:39:01.895" pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1599vm-test-run-nixbot-gitlab> gitlab # [  201.013911] gitaly[1392]: time="2026-07-04T12:39:01.923Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=329 command.oublock=32 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7BTK51X1250TDH00879N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-04T12:39:11.155" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T12:39:01.895" grpc.time_ms=27.384 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1600vm-test-run-nixbot-gitlab> gitlab # [  201.078797] gitaly[1392]: time="2026-07-04T12:39:02.030Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7BTK51X1250TDH00879N grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-04T12:39:12.008" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T12:39:02.029" pid=1392 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1601vm-test-run-nixbot-gitlab> gitlab # [  201.080128] gitaly[1392]: time="2026-07-04T12:39:02.030Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7BTK51X1250TDH00879N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-04T12:39:12.008" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T12:39:02.029" grpc.time_ms=1.163 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1602vm-test-run-nixbot-gitlab> gitlab # [  201.121376] gitaly[1392]: time="2026-07-04T12:39:02.072Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7BTK51X1250TDH00879N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-04T12:39:12.044" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T12:39:02.065" grpc.time_ms=7.798 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1603vm-test-run-nixbot-gitlab> gitlab # [  201.144511] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:02.095Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"ec373b0fdcbf9d29466793d4","created_at":"2026-07-04T12:39:02.084Z","trace_propagation_headers":{"sentry-trace":"081a4b7bd03f491591eef04390a63f64-0744ec62960e4f72","baggage":"sentry-trace_id=081a4b7bd03f491591eef04390a63f64,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPJ7BTK51X1250TDH00879N","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:02.092Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-ec373b0fdcbf9d29466793d4: start","job_status":"start","queue_duration_s":0.002517,"scheduling_latency_s":0.002591}1604vm-test-run-nixbot-gitlab> gitlab # [  201.159243] gitaly[1392]: time="2026-07-04T12:39:02.110Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7BTK51X1250TDH00879N error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-04T12:39:12.087" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T12:39:02.108" pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1605vm-test-run-nixbot-gitlab> gitlab # [  201.161218] gitaly[1392]: time="2026-07-04T12:39:02.110Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7BTK51X1250TDH00879N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-04T12:39:12.087" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T12:39:02.108" grpc.time_ms=1.701 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1606vm-test-run-nixbot-gitlab> gitlab # [  201.225882] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:02.176Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"ec373b0fdcbf9d29466793d4","created_at":"2026-07-04T12:39:02.084Z","trace_propagation_headers":{"sentry-trace":"081a4b7bd03f491591eef04390a63f64-0744ec62960e4f72","baggage":"sentry-trace_id=081a4b7bd03f491591eef04390a63f64,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPJ7BTK51X1250TDH00879N","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:02.092Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-ec373b0fdcbf9d29466793d4: done: 0.08139 sec","job_status":"done","queue_duration_s":0.002517,"scheduling_latency_s":0.002591,"redis_calls":10,"redis_duration_s":0.004457,"redis_read_bytes":8,"redis_write_bytes":1165,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002427,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00203,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":7,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040699,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.046284094,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.08139,"completed_at":"2026-07-04T12:39:02.176Z","load_balancing_strategy":"primary","db_duration_s":0.015686,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1607vm-test-run-nixbot-gitlab> gitlab # [  201.264334] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:02.178Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"ee0231a1dad96ae906f1d2b8","created_at":"2026-07-04T12:39:02.098Z","trace_propagation_headers":{"sentry-trace":"081a4b7bd03f491591eef04390a63f64-0744ec62960e4f72","baggage":"sentry-trace_id=081a4b7bd03f491591eef04390a63f64,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPJ7BTK51X1250TDH00879N","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:02.103Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-ee0231a1dad96ae906f1d2b8: start","job_status":"start","queue_duration_s":0.074894,"scheduling_latency_s":0.074921}1608vm-test-run-nixbot-gitlab> gitlab # [  201.267214] gitaly[1392]: time="2026-07-04T12:39:02.188Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7BTK51X1250TDH00879N diskcache=8c020a38-4e38-4872-a4b9-b50b8e0e9082 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-04T12:39:12.093" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T12:39:02.114" pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1609vm-test-run-nixbot-gitlab> gitlab # [  201.268615] gitaly[1392]: time="2026-07-04T12:39:02.188Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=329 command.oublock=32 command.real_time_ms=40 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7BTK51X1250TDH00879N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-04T12:39:12.093" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T12:39:02.114" grpc.time_ms=74.131 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1610vm-test-run-nixbot-gitlab> gitlab # [  201.286793] gitaly[1392]: time="2026-07-04T12:39:02.203Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7BTK51X1250TDH00879N grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-04T12:39:12.182" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T12:39:02.203" pid=1392 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1611vm-test-run-nixbot-gitlab> gitlab # [  201.359293] gitaly[1392]: time="2026-07-04T12:39:02.203Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7BTK51X1250TDH00879N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-04T12:39:12.182" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T12:39:02.203" grpc.time_ms=0.412 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1612vm-test-run-nixbot-gitlab> gitlab # [  201.360884] gitaly[1392]: time="2026-07-04T12:39:02.223Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7BTK51X1250TDH00879N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-04T12:39:12.188" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T12:39:02.209" grpc.time_ms=14.257 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1613vm-test-run-nixbot-gitlab> gitlab # [  201.362047] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:02.269Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"ee0231a1dad96ae906f1d2b8","created_at":"2026-07-04T12:39:02.098Z","trace_propagation_headers":{"sentry-trace":"081a4b7bd03f491591eef04390a63f64-0744ec62960e4f72","baggage":"sentry-trace_id=081a4b7bd03f491591eef04390a63f64,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPJ7BTK51X1250TDH00879N","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:02.103Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-ee0231a1dad96ae906f1d2b8: done: 0.091262 sec","job_status":"done","queue_duration_s":0.074894,"scheduling_latency_s":0.074921,"redis_calls":14,"redis_duration_s":0.0034260000000000002,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000696,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00163,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.0011,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.060903,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.066880288,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.091262,"completed_at":"2026-07-04T12:39:02.269Z","load_balancing_strategy":"primary","db_duration_s":0.019207,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1614vm-test-run-nixbot-gitlab> gitlab # [  201.431454] gitaly[1392]: time="2026-07-04T12:39:02.251Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7BTK51X1250TDH00879N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-04T12:39:12.230" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T12:39:02.250" grpc.time_ms=0.667 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1615vm-test-run-nixbot-gitlab> gitlab # [  201.432892] gitaly[1392]: time="2026-07-04T12:39:02.256Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7BTK51X1250TDH00879N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-04T12:39:12.235" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T12:39:02.256" grpc.time_ms=0.186 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1616vm-test-run-nixbot-gitlab> gitlab # [  201.732226] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:02.683Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"688712a6affd0e0ca6c1176f","created_at":"2026-07-04T12:39:02.672Z","trace_propagation_headers":{"sentry-trace":"081a4b7bd03f491591eef04390a63f64-0744ec62960e4f72","baggage":"sentry-trace_id=081a4b7bd03f491591eef04390a63f64,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPJ7BTK51X1250TDH00879N","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:02.679Z","job_size_bytes":5,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-688712a6affd0e0ca6c1176f: start","job_status":"start","queue_duration_s":0.002986,"scheduling_latency_s":0.003058}1617vm-test-run-nixbot-gitlab> gitlab # [  201.906167] gitaly[1392]: time="2026-07-04T12:39:02.857Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7BTK51X1250TDH00879N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-04T12:39:12.105" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T12:39:02.846" grpc.time_ms=11.138 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1618vm-test-run-nixbot-gitlab> gitlab # [  201.951224] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:02.902Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"688712a6affd0e0ca6c1176f","created_at":"2026-07-04T12:39:02.672Z","trace_propagation_headers":{"sentry-trace":"081a4b7bd03f491591eef04390a63f64-0744ec62960e4f72","baggage":"sentry-trace_id=081a4b7bd03f491591eef04390a63f64,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPJ7BTK51X1250TDH00879N","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:02.679Z","job_size_bytes":5,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-688712a6affd0e0ca6c1176f: done: 0.219404 sec","job_status":"done","queue_duration_s":0.002986,"scheduling_latency_s":0.003058,"redis_calls":12,"redis_duration_s":0.007148,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.002072,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003841,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001235,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.105326,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000827759,"duration_s":0.219404,"completed_at":"2026-07-04T12:39:02.902Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.026379,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1619vm-test-run-nixbot-gitlab> gitlab # [  201.964345] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:02.904Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"794e8b1e48868d969bf25f2e","created_at":"2026-07-04T12:39:02.798Z","trace_propagation_headers":{"sentry-trace":"081a4b7bd03f491591eef04390a63f64-0744ec62960e4f72","baggage":"sentry-trace_id=081a4b7bd03f491591eef04390a63f64,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPJ7BTK51X1250TDH00879N","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:02.808Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-794e8b1e48868d969bf25f2e: start","job_status":"start","queue_duration_s":0.09571,"scheduling_latency_s":0.095752}1620vm-test-run-nixbot-gitlab> gitlab # [  202.114267] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:03.065Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"794e8b1e48868d969bf25f2e","created_at":"2026-07-04T12:39:02.798Z","trace_propagation_headers":{"sentry-trace":"081a4b7bd03f491591eef04390a63f64-0744ec62960e4f72","baggage":"sentry-trace_id=081a4b7bd03f491591eef04390a63f64,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPJ7BTK51X1250TDH00879N","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:02.808Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-794e8b1e48868d969bf25f2e: done: 0.160948 sec","job_status":"done","queue_duration_s":0.09571,"scheduling_latency_s":0.095752,"redis_calls":11,"redis_duration_s":0.003779,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.00145,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001566,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000763,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.122,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.122,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.103327,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000361219,"duration_s":0.160948,"completed_at":"2026-07-04T12:39:03.065Z","load_balancing_strategy":"primary","db_duration_s":0.051013,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1621vm-test-run-nixbot-gitlab> gitlab # [  202.116396] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:03.067Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"b74fc0991532702cbc5ad472","created_at":"2026-07-04T12:39:02.815Z","trace_propagation_headers":{"sentry-trace":"081a4b7bd03f491591eef04390a63f64-0744ec62960e4f72","baggage":"sentry-trace_id=081a4b7bd03f491591eef04390a63f64,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPJ7BTK51X1250TDH00879N","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:02.824Z","job_size_bytes":89,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-b74fc0991532702cbc5ad472: start","job_status":"start","queue_duration_s":0.242488,"scheduling_latency_s":0.24251}1622vm-test-run-nixbot-gitlab> gitlab # [  202.172528] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:03.123Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"b74fc0991532702cbc5ad472","created_at":"2026-07-04T12:39:02.815Z","trace_propagation_headers":{"sentry-trace":"081a4b7bd03f491591eef04390a63f64-0744ec62960e4f72","baggage":"sentry-trace_id=081a4b7bd03f491591eef04390a63f64,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPJ7BTK51X1250TDH00879N","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:02.824Z","job_size_bytes":89,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-b74fc0991532702cbc5ad472: done: 0.056587 sec","job_status":"done","queue_duration_s":0.242488,"scheduling_latency_s":0.24251,"redis_calls":11,"redis_duration_s":0.002624,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000659,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001463,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000502,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029553,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000373232,"duration_s":0.056587,"completed_at":"2026-07-04T12:39:03.123Z","load_balancing_strategy":"primary","db_duration_s":0.021084,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1623vm-test-run-nixbot-gitlab> gitlab # [  202.244560] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:03 +0000] "POST /api/v4/projects HTTP/1.1" 201 4795 "" "curl/8.20.0" 19921624vm-test-run-nixbot-gitlab> gitlab # [  202.258376] setup-gitlab-start[2696]: + mkdir -p /tmp/test-flake1625vm-test-run-nixbot-gitlab> gitlab # [  202.260105] setup-gitlab-start[2696]: + cd /tmp/test-flake1626vm-test-run-nixbot-gitlab> gitlab # [  202.261154] setup-gitlab-start[2696]: + git init -b master1627vm-test-run-nixbot-gitlab> gitlab # [  202.277243] setup-gitlab-start[3013]: Initialized empty Git repository in /tmp/test-flake/.git/1628vm-test-run-nixbot-gitlab> gitlab # [  202.278237] setup-gitlab-start[2696]: + git config user.name test1629vm-test-run-nixbot-gitlab> gitlab # [  202.279283] setup-gitlab-start[2696]: + git config user.email test@example.com1630vm-test-run-nixbot-gitlab> gitlab # [  202.281202] setup-gitlab-start[2696]: + cat1631vm-test-run-nixbot-gitlab> gitlab # [  202.282966] setup-gitlab-start[2696]: + git add flake.nix1632vm-test-run-nixbot-gitlab> gitlab # [  202.285637] setup-gitlab-start[2696]: + git commit -m 'initial commit'1633vm-test-run-nixbot-gitlab> gitlab # [  202.293057] setup-gitlab-start[3018]: [master (root-commit) fd1fc7e] initial commit1634vm-test-run-nixbot-gitlab> gitlab # [  202.294366] setup-gitlab-start[3018]:  1 file changed, 10 insertions(+)1635vm-test-run-nixbot-gitlab> gitlab # [  202.295507] setup-gitlab-start[3018]:  create mode 100644 flake.nix1636vm-test-run-nixbot-gitlab> gitlab # [  202.296378] setup-gitlab-start[2696]: + git remote add origin http://oauth2:glpat-chNOUPOfvDkXEMZZsZAOy286MQp1OjEH.01.0w02nn868@gitlab/root/test-flake.git1637vm-test-run-nixbot-gitlab> gitlab # [  202.298800] setup-gitlab-start[2696]: + git push -u origin master1638vm-test-run-nixbot-gitlab> gitlab # [  202.362180] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:03 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 441639vm-test-run-nixbot-gitlab> gitlab # [  202.460067] gitaly[1392]: time="2026-07-04T12:39:03.410Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7DWSHFYEYKZMNCKF45Y1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T12:39:03.396" grpc.time_ms=13.695 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1640vm-test-run-nixbot-gitlab> gitlab # [  202.463792] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:03 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.54.0" 931641vm-test-run-nixbot-gitlab> gitlab # [  202.706470] gitaly[1392]: time="2026-07-04T12:39:03.657Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=621 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-04T12:39:13.009" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T12:39:03.649" grpc.time_ms=7.528 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1642vm-test-run-nixbot-gitlab> gitlab # [  202.778615] gitaly[1392]: time="2026-07-04T12:39:03.730Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=331 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-07-04T12:39:33.020" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-07-04T12:39:03.721" grpc.time_ms=8.689 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1643vm-test-run-nixbot-gitlab> gitlab # [  202.821682] gitaly[1392]: time="2026-07-04T12:39:03.773Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G duration_ms=173ns method=POST pid=1392 status=200 url="http://unix/api/v4/internal/allowed"1644vm-test-run-nixbot-gitlab> gitlab # [  202.846834] gitaly[1392]: time="2026-07-04T12:39:03.798Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G duration_ms=23ns method=POST pid=1392 status=200 url="http://unix/api/v4/internal/pre_receive"1645vm-test-run-nixbot-gitlab> gitlab # [  202.856343] gitaly[1392]: time="2026-07-04T12:39:03.807Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5322 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T12:39:03.599" grpc.time_ms=208.386 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1646vm-test-run-nixbot-gitlab> gitlab # [  202.900156] gitaly[1392]: time="2026-07-04T12:39:03.851Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4792 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T12:39:03.851" grpc.time_ms=0.593 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1647vm-test-run-nixbot-gitlab> gitlab # [  202.945683] gitaly[1392]: time="2026-07-04T12:39:03.897Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4791 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T12:39:03.896" grpc.time_ms=0.547 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1648vm-test-run-nixbot-gitlab> gitlab # [  202.989385] gitaly[1392]: time="2026-07-04T12:39:03.940Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4793 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T12:39:03.940" grpc.time_ms=0.38 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1649vm-test-run-nixbot-gitlab> gitlab # [  203.520574] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:04.470Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 fd1fc7ecbe895687b4904ab556cca5a4e96ea8fb refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"97300ad4c6a22839e7afe189","created_at":"2026-07-04T12:39:04.437Z","trace_propagation_headers":{"sentry-trace":"d311a4d5f6d54ca6a50d968410119c29-cd3eaa9fe1b34a54","baggage":"sentry-trace_id=d311a4d5f6d54ca6a50d968410119c29,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPJ7E1BWTSTSTFPBQNWTB5G","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.466Z","job_size_bytes":185,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-97300ad4c6a22839e7afe189: start","job_status":"start","queue_duration_s":0.004515,"scheduling_latency_s":0.004559}1650vm-test-run-nixbot-gitlab> gitlab # [  203.539343] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:04.490Z","message":"changes: \"0000000000000000000000000000000000000000 fd1fc7ecbe895687b4904ab556cca5a4e96ea8fb refs/heads/master\\n\""}1651vm-test-run-nixbot-gitlab> gitlab # [  203.606964] gitaly[1392]: time="2026-07-04T12:39:04.558Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-04T12:39:14.001" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T12:39:04.551" grpc.time_ms=6.442 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1652vm-test-run-nixbot-gitlab> gitlab # [  203.637117] gitaly[1392]: time="2026-07-04T12:39:04.588Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-04T12:39:14.030" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T12:39:04.580" grpc.time_ms=7.461 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1653vm-test-run-nixbot-gitlab> gitlab # [  203.661228] gitaly[1392]: time="2026-07-04T12:39:04.612Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1563 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=6 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-07-04T12:39:14.042" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T12:39:04.593" grpc.time_ms=19.515 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1654vm-test-run-nixbot-gitlab> gitlab # [  203.682899] gitaly[1392]: time="2026-07-04T12:39:04.634Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-04T12:39:14.005" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T12:39:04.606" grpc.time_ms=6.144 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1655vm-test-run-nixbot-gitlab> gitlab # [  203.714124] gitaly[1392]: time="2026-07-04T12:39:04.665Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-07-04T12:39:14.106" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=226 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T12:39:04.657" grpc.time_ms=8.497 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1656vm-test-run-nixbot-gitlab> gitlab # [  203.764089] gitaly[1392]: time="2026-07-04T12:39:04.715Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3942 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T12:39:04.715" grpc.time_ms=0.389 pid=1392 span.kind=server system=grpc1657vm-test-run-nixbot-gitlab> gitlab # [  203.780507] gitaly[1392]: time="2026-07-04T12:39:04.732Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G duration_ms=746ns method=POST pid=1392 status=200 url="http://unix/api/v4/internal/post_receive"1658vm-test-run-nixbot-gitlab> gitlab # [  203.781966] gitaly[1392]: time="2026-07-04T12:39:04.733Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4816 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T12:39:03.984" grpc.time_ms=748.688 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1659vm-test-run-nixbot-gitlab> gitlab # [  203.812976] gitaly[1392]: time="2026-07-04T12:39:04.763Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G diskcache=c21dbdd8-12d1-4a3a-a1bc-ba0c55e8c352 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T12:39:03.523" pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1660vm-test-run-nixbot-gitlab> gitlab # [  203.814308] gitaly[1392]: time="2026-07-04T12:39:04.764Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=235 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13817 command.oublock=32 command.real_time_ms=1229 command.spawn_token_fork_ms=1 command.system_time_ms=169 command.user_time_ms=66 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T12:39:03.523" grpc.time_ms=1240.169 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1661vm-test-run-nixbot-gitlab> gitlab # [  203.818471] gitlab-workhorse[1293]: time="2026-07-04T12:39:04Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521662vm-test-run-nixbot-gitlab> gitlab # [  203.821130] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:04 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 13011663vm-test-run-nixbot-gitlab> gitlab # [  203.857386] setup-gitlab-start[3022]: To http://gitlab/root/test-flake.git1664vm-test-run-nixbot-gitlab> gitlab # [  203.858843] setup-gitlab-start[3022]:  * [new branch]      master -> master1665vm-test-run-nixbot-gitlab> gitlab # [  203.862383] setup-gitlab-start[3022]: branch 'master' set up to track 'origin/master'.1666vm-test-run-nixbot-gitlab> gitlab # [  203.864828] systemd[1]: Finished setup-gitlab.service.1667vm-test-run-nixbot-gitlab> gitlab # [  203.869248] systemd[1]: Starting nixbot CI service...1668vm-test-run-nixbot-gitlab> gitlab # [  203.871196] gitaly[1392]: time="2026-07-04T12:39:04.821Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3944 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T12:39:04.820" grpc.time_ms=0.411 pid=1392 span.kind=server system=grpc1669vm-test-run-nixbot-gitlab> gitlab # [  203.896353] gitaly[1392]: time="2026-07-04T12:39:04.847Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G diskcache=03b6be88-41ee-4938-9cff-513ffcc0cfef grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-04T12:39:14.137" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T12:39:04.687" pid=1392 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1670vm-test-run-nixbot-gitlab> gitlab # [  203.913261] gitaly[1392]: time="2026-07-04T12:39:04.864Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=75 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4850 command.oublock=8 command.real_time_ms=150 command.spawn_token_fork_ms=0 command.system_time_ms=57 command.user_time_ms=18 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-04T12:39:14.137" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T12:39:04.687" grpc.time_ms=160.07 pid=1392 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1671vm-test-run-nixbot-gitlab> gitlab # [  203.978288] gitaly[1392]: time="2026-07-04T12:39:04.929Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-04T12:39:14.008" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T12:39:04.928" grpc.time_ms=0.159 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1672vm-test-run-nixbot-gitlab> gitlab # [  204.007154] gitaly[1392]: time="2026-07-04T12:39:04.957Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=942 command.oublock=0 command.real_time_ms=20 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-07-04T12:39:14.014" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T12:39:04.935" grpc.time_ms=22.048 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1673vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for unit setup-gitlab.service, in 205.05 seconds)1674vm-test-run-nixbot-gitlab> gitlab # [  204.123148] gitaly[1392]: time="2026-07-04T12:39:05.074Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=343 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-07-04T12:39:35.069" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:05.069" grpc.time_ms=4.662 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1675vm-test-run-nixbot-gitlab> gitlab: waiting for unit nixbot.service1676vm-test-run-nixbot-gitlab> gitlab # [  204.179933] gitaly[1392]: time="2026-07-04T12:39:05.131Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-04T12:39:15.010" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T12:39:05.130" grpc.time_ms=0.436 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1677vm-test-run-nixbot-gitlab> gitlab # [  204.199056] gitaly[1392]: time="2026-07-04T12:39:05.139Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=343 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-07-04T12:39:15.013" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-07-04T12:39:05.134" grpc.time_ms=5.348 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1678vm-test-run-nixbot-gitlab> gitlab # [  204.212755] gitaly[1392]: time="2026-07-04T12:39:05.164Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-04T12:39:15.036" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T12:39:05.156" grpc.time_ms=7.209 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1679vm-test-run-nixbot-gitlab> gitlab # [  204.225234] gitaly[1392]: time="2026-07-04T12:39:05.176Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-04T12:39:15.049" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T12:39:05.169" grpc.time_ms=7.146 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1680vm-test-run-nixbot-gitlab> gitlab # [  204.249513] gitaly[1392]: time="2026-07-04T12:39:05.201Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=960 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-04T12:39:35.187" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:05.187" grpc.time_ms=13.582 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1681vm-test-run-nixbot-gitlab> gitlab # [  204.551418] gitaly[1392]: time="2026-07-04T12:39:05.502Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-07-04T12:39:15.010" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:05.500" grpc.time_ms=1.835 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1682vm-test-run-nixbot-gitlab> gitlab # [  204.589140] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:05.539Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 fd1fc7ecbe895687b4904ab556cca5a4e96ea8fb refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"97300ad4c6a22839e7afe189","created_at":"2026-07-04T12:39:04.437Z","trace_propagation_headers":{"sentry-trace":"d311a4d5f6d54ca6a50d968410119c29-cd3eaa9fe1b34a54","baggage":"sentry-trace_id=d311a4d5f6d54ca6a50d968410119c29,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPJ7E1BWTSTSTFPBQNWTB5G","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.466Z","job_size_bytes":185,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-97300ad4c6a22839e7afe189: done: 1.069401 sec","job_status":"done","queue_duration_s":0.004515,"scheduling_latency_s":0.004559,"gitaly_calls":14,"gitaly_duration_s":0.39565,"redis_calls":201,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.106141,"redis_read_bytes":924,"redis_write_bytes":19883,"redis_cache_calls":14,"redis_cache_duration_s":0.008272,"redis_cache_read_bytes":16,"redis_cache_write_bytes":884,"redis_feature_flag_calls":32,"redis_feature_flag_duration_s":0.010352,"redis_feature_flag_read_bytes":540,"redis_feature_flag_write_bytes":4279,"redis_queues_calls":8,"redis_queues_duration_s":0.001631,"redis_queues_read_bytes":10,"redis_queues_write_bytes":4934,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.003676,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1259,"redis_repository_cache_calls":92,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.067259,"redis_repository_cache_read_bytes":214,"redis_repository_cache_write_bytes":6392,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.014951,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":51,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":51,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.057,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":51,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.197,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.238,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.057,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.470249,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00078781,"exclusive_lock_hold_duration_s":0.028338518,"duration_s":1.069401,"completed_at":"2026-07-04T12:39:05.539Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.122029,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1683vm-test-run-nixbot-gitlab> gitlab # [  204.668421] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:05.541Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"0c70f72c60a67efaca3db82a","created_at":"2026-07-04T12:39:04.622Z","trace_propagation_headers":{"sentry-trace":"d311a4d5f6d54ca6a50d968410119c29-d153f669afdb4c15","baggage":"sentry-trace_id=d311a4d5f6d54ca6a50d968410119c29,sentry-release=Unknown"},"correlation_id":"01KWPJ7E1BWTSTSTFPBQNWTB5G","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.627Z","job_size_bytes":46,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-0c70f72c60a67efaca3db82a: start","job_status":"start","queue_duration_s":0.914305,"scheduling_latency_s":0.914327}1684vm-test-run-nixbot-gitlab> gitlab # [  204.713416] gitaly[1392]: time="2026-07-04T12:39:05.664Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-07-04T18:39:05.661" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T12:39:05.661" grpc.time_ms=2.466 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1685vm-test-run-nixbot-gitlab> gitlab # [  204.733745] gitaly[1392]: time="2026-07-04T12:39:05.685Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=649 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-07-04T12:39:35.676" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:05.677" grpc.time_ms=7.822 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1686vm-test-run-nixbot-gitlab> gitlab # [  204.758762] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:05.710Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"0c70f72c60a67efaca3db82a","created_at":"2026-07-04T12:39:04.622Z","trace_propagation_headers":{"sentry-trace":"d311a4d5f6d54ca6a50d968410119c29-d153f669afdb4c15","baggage":"sentry-trace_id=d311a4d5f6d54ca6a50d968410119c29,sentry-release=Unknown"},"correlation_id":"01KWPJ7E1BWTSTSTFPBQNWTB5G","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.627Z","job_size_bytes":46,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-0c70f72c60a67efaca3db82a: done: 0.168312 sec","job_status":"done","queue_duration_s":0.914305,"scheduling_latency_s":0.914327,"gitaly_calls":2,"gitaly_duration_s":0.094525,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.008678,"redis_read_bytes":69,"redis_write_bytes":4889,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001369,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.00053,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2116,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001764,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.004056,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":754,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000959,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053282,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000341385,"duration_s":0.168312,"completed_at":"2026-07-04T12:39:05.709Z","load_balancing_strategy":"primary","db_duration_s":0.013714,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1687vm-test-run-nixbot-gitlab> gitlab # [  204.797550] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:05.711Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"7f778065cfd28d52d2b6f2a7","created_at":"2026-07-04T12:39:04.637Z","trace_propagation_headers":{"sentry-trace":"d311a4d5f6d54ca6a50d968410119c29-d153f669afdb4c15","baggage":"sentry-trace_id=d311a4d5f6d54ca6a50d968410119c29,sentry-release=Unknown"},"correlation_id":"01KWPJ7E1BWTSTSTFPBQNWTB5G","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.641Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-7f778065cfd28d52d2b6f2a7: start","job_status":"start","queue_duration_s":1.07046,"scheduling_latency_s":1.070482}1688vm-test-run-nixbot-gitlab> gitlab # [  204.807561] gitaly[1392]: time="2026-07-04T12:39:05.759Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G diskcache=4720baf6-f70c-4484-90af-bf982c0741c1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-04T18:39:05.733" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:05.734" pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1689vm-test-run-nixbot-gitlab> gitlab # [  204.808562] gitaly[1392]: time="2026-07-04T12:39:05.759Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1287 command.oublock=0 command.real_time_ms=23 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-04T18:39:05.733" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:05.734" grpc.time_ms=25.036 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1690vm-test-run-nixbot-gitlab> gitlab # [  204.934559] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:05.885Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"7f778065cfd28d52d2b6f2a7","created_at":"2026-07-04T12:39:04.637Z","trace_propagation_headers":{"sentry-trace":"d311a4d5f6d54ca6a50d968410119c29-d153f669afdb4c15","baggage":"sentry-trace_id=d311a4d5f6d54ca6a50d968410119c29,sentry-release=Unknown"},"correlation_id":"01KWPJ7E1BWTSTSTFPBQNWTB5G","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.641Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-7f778065cfd28d52d2b6f2a7: done: 0.173828 sec","job_status":"done","queue_duration_s":1.07046,"scheduling_latency_s":1.070482,"gitaly_calls":1,"gitaly_duration_s":0.027682,"redis_calls":15,"redis_duration_s":0.004714,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000444,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001924,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000974,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001372,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.095,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.096,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.104,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.095,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037818,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.147207257,"duration_s":0.173828,"completed_at":"2026-07-04T12:39:05.885Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.101352,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1691vm-test-run-nixbot-gitlab> gitlab # [  204.997225] nixbot[3144]: {"ts": "2026-07-04T12:39:05.947Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1692vm-test-run-nixbot-gitlab> gitlab # [  205.020493] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:05.971Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"49bdc6dff5799120c77cc9f8","created_at":"2026-07-04T12:39:04.790Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1b1180c65e08a5aaf31d85c5f1d85d4f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.796Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-49bdc6dff5799120c77cc9f8: start","job_status":"start","queue_duration_s":1.174754,"scheduling_latency_s":1.174826}1693vm-test-run-nixbot-gitlab> gitlab # [  205.028890] nixbot[3144]: {"ts": "2026-07-04T12:39:05.980Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1694vm-test-run-nixbot-gitlab> gitlab # [  205.054551] nixbot[3144]: {"ts": "2026-07-04T12:39:06.005Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1695vm-test-run-nixbot-gitlab> gitlab # [  205.063358] nixbot[3144]: {"ts": "2026-07-04T12:39:06.014Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1696vm-test-run-nixbot-gitlab> gitlab # [  205.067248] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:06.015Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"49bdc6dff5799120c77cc9f8","created_at":"2026-07-04T12:39:04.790Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1b1180c65e08a5aaf31d85c5f1d85d4f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.796Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-49bdc6dff5799120c77cc9f8: done: 0.043966 sec","job_status":"done","queue_duration_s":1.174754,"scheduling_latency_s":1.174826,"redis_calls":18,"redis_duration_s":0.006467,"redis_read_bytes":651,"redis_write_bytes":1923,"redis_feature_flag_calls":11,"redis_feature_flag_duration_s":0.003646,"redis_feature_flag_read_bytes":646,"redis_feature_flag_write_bytes":1147,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001923,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000898,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020985,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.043966,"completed_at":"2026-07-04T12:39:06.015Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00479,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1697vm-test-run-nixbot-gitlab> gitlab # [  205.105042] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:06.017Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"6f464982213e6fca535d9205","created_at":"2026-07-04T12:39:04.800Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"417ece4f80f13de3f21df078a72a50cc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.802Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-6f464982213e6fca535d9205: start","job_status":"start","queue_duration_s":1.214813,"scheduling_latency_s":1.214842}1698vm-test-run-nixbot-gitlab> gitlab # [  205.105759] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:06.039Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"6f464982213e6fca535d9205","created_at":"2026-07-04T12:39:04.800Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"417ece4f80f13de3f21df078a72a50cc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.802Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-6f464982213e6fca535d9205: done: 0.02223 sec","job_status":"done","queue_duration_s":1.214813,"scheduling_latency_s":1.214842,"redis_calls":7,"redis_duration_s":0.002709,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001895,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000814,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014234,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02223,"completed_at":"2026-07-04T12:39:06.039Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1699vm-test-run-nixbot-gitlab> gitlab # [  205.189310] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:06.041Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"d9fbdeb83dd013c66b0b573d","created_at":"2026-07-04T12:39:04.806Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a52968b339ec6921501bf63bea3eaec8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.808Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-d9fbdeb83dd013c66b0b573d: start","job_status":"start","queue_duration_s":1.233062,"scheduling_latency_s":1.233085}1700vm-test-run-nixbot-gitlab> gitlab # [  205.190853] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:06.048Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1701vm-test-run-nixbot-gitlab> gitlab # [  205.191699] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:06.053Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"d9fbdeb83dd013c66b0b573d","created_at":"2026-07-04T12:39:04.806Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a52968b339ec6921501bf63bea3eaec8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.808Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-d9fbdeb83dd013c66b0b573d: done: 0.012226 sec","job_status":"done","queue_duration_s":1.233062,"scheduling_latency_s":1.233085,"redis_calls":7,"redis_duration_s":0.0016489999999999999,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00114,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000509,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007234,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012226,"completed_at":"2026-07-04T12:39:06.053Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1702vm-test-run-nixbot-gitlab> gitlab # [  205.276137] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:06.055Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"2f6f25bae09738efc524c6df","created_at":"2026-07-04T12:39:04.814Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d4339f88623ee52d58d0227f277f41ad","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.816Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-2f6f25bae09738efc524c6df: start","job_status":"start","queue_duration_s":1.238402,"scheduling_latency_s":1.238423}1703vm-test-run-nixbot-gitlab> gitlab # [  205.277627] nixbot[3144]: {"ts": "2026-07-04T12:39:06.021Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1704vm-test-run-nixbot-gitlab> gitlab # [  205.278902] nixbot[3144]: {"ts": "2026-07-04T12:39:06.029Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1705vm-test-run-nixbot-gitlab> gitlab # [  205.280835] nixbot[3144]: {"ts": "2026-07-04T12:39:06.033Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1706vm-test-run-nixbot-gitlab> gitlab # [  205.282235] nixbot[3144]: {"ts": "2026-07-04T12:39:06.036Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1707vm-test-run-nixbot-gitlab> gitlab # [  205.283416] nixbot[3144]: {"ts": "2026-07-04T12:39:06.042Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1708vm-test-run-nixbot-gitlab> gitlab # [  205.284863] nixbot[3144]: {"ts": "2026-07-04T12:39:06.044Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1709vm-test-run-nixbot-gitlab> gitlab # [  205.286423] nixbot[3144]: {"ts": "2026-07-04T12:39:06.048Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1710vm-test-run-nixbot-gitlab> gitlab # [  205.289125] nixbot[3144]: {"ts": "2026-07-04T12:39:06.053Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1711vm-test-run-nixbot-gitlab> gitlab # [  205.292542] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:06.243Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"2f6f25bae09738efc524c6df","created_at":"2026-07-04T12:39:04.814Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d4339f88623ee52d58d0227f277f41ad","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.816Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-2f6f25bae09738efc524c6df: done: 0.188774 sec","job_status":"done","queue_duration_s":1.238402,"scheduling_latency_s":1.238423,"redis_calls":12,"redis_duration_s":0.004738,"redis_read_bytes":137,"redis_write_bytes":1329,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001404,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":342,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001603,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001731,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":124,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":124,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.032,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":123,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":80,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.031,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.14198,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000500064,"exclusive_lock_hold_duration_s":0.168503259,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.188774,"completed_at":"2026-07-04T12:39:06.243Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.052862,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1712vm-test-run-nixbot-gitlab> gitlab # [  205.383363] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:06.245Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"aa5070fbb1d67f69ddbecd52","created_at":"2026-07-04T12:39:04.821Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"532c60ed945aa51cefd0e7bf816c9e55","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.823Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-aa5070fbb1d67f69ddbecd52: start","job_status":"start","queue_duration_s":1.422404,"scheduling_latency_s":1.422428}1713vm-test-run-nixbot-gitlab> gitlab # [  205.385325] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:06.298Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"aa5070fbb1d67f69ddbecd52","created_at":"2026-07-04T12:39:04.821Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"532c60ed945aa51cefd0e7bf816c9e55","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.823Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-aa5070fbb1d67f69ddbecd52: done: 0.052179 sec","job_status":"done","queue_duration_s":1.422404,"scheduling_latency_s":1.422428,"redis_calls":9,"redis_duration_s":0.004063,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001972,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002091,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025132,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001482032,"exclusive_lock_hold_duration_s":0.019609755,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.052179,"completed_at":"2026-07-04T12:39:06.298Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.019838,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1714vm-test-run-nixbot-gitlab> gitlab # [  205.460338] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:06.300Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"7a76b3dd8cf5d0cbdeaaf94a","created_at":"2026-07-04T12:39:04.834Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bee46d1305b4642750997ba659354313","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.836Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-7a76b3dd8cf5d0cbdeaaf94a: start","job_status":"start","queue_duration_s":1.463585,"scheduling_latency_s":1.463626}1715vm-test-run-nixbot-gitlab> gitlab # [  205.545772] nixbot[3144]: {"ts": "2026-07-04T12:39:06.497Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1716vm-test-run-nixbot-gitlab> gitlab # [  205.549058] nixbot[3144]: {"ts": "2026-07-04T12:39:06.500Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1717vm-test-run-nixbot-gitlab> gitlab # [  205.552658] nixbot[3144]: {"ts": "2026-07-04T12:39:06.504Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1718vm-test-run-nixbot-gitlab> gitlab # [  205.554702] nixbot[3144]: {"ts": "2026-07-04T12:39:06.506Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1719vm-test-run-nixbot-gitlab> gitlab # [  205.556738] nixbot[3144]: {"ts": "2026-07-04T12:39:06.508Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1720vm-test-run-nixbot-gitlab> gitlab # [  205.558676] nixbot[3144]: {"ts": "2026-07-04T12:39:06.510Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1721vm-test-run-nixbot-gitlab> gitlab # [  205.562398] nixbot[3144]: {"ts": "2026-07-04T12:39:06.514Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1722vm-test-run-nixbot-gitlab> gitlab # [  205.564449] nixbot[3144]: {"ts": "2026-07-04T12:39:06.516Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"}1723vm-test-run-nixbot-gitlab> gitlab # [  205.566380] nixbot[3144]: {"ts": "2026-07-04T12:39:06.517Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"}1724vm-test-run-nixbot-gitlab> gitlab # [  205.872719] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:06.823Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"7a76b3dd8cf5d0cbdeaaf94a","created_at":"2026-07-04T12:39:04.834Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bee46d1305b4642750997ba659354313","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.836Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-7a76b3dd8cf5d0cbdeaaf94a: done: 0.523037 sec","job_status":"done","queue_duration_s":1.463585,"scheduling_latency_s":1.463626,"redis_calls":571,"redis_duration_s":0.17295,"redis_read_bytes":569,"redis_write_bytes":49487,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001279,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.171671,"redis_shared_state_read_bytes":565,"redis_shared_state_write_bytes":49017,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.373517,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.523037,"completed_at":"2026-07-04T12:39:06.823Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1725vm-test-run-nixbot-gitlab> gitlab # [  205.893930] nixbot[3144]: INFO:     Started server process [3144]1726vm-test-run-nixbot-gitlab> gitlab # [  205.898240] nixbot[3144]: INFO:     Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)1727vm-test-run-nixbot-gitlab> gitlab # [  205.899985] nixbot[3144]: INFO:     ::ffff:127.0.0.1:42674 - "GET /health HTTP/1.1" 200 OK1728vm-test-run-nixbot-gitlab> gitlab # [  205.978432] systemd[1]: Started nixbot CI service.1729vm-test-run-nixbot-gitlab> gitlab # [  205.982100] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:06.930Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"bc497f262a01911a83034caa","created_at":"2026-07-04T12:39:04.841Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e762bbfa5ceab4c19004eebc5d6c3256","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.843Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-bc497f262a01911a83034caa: start","job_status":"start","queue_duration_s":2.086674,"scheduling_latency_s":2.08672}1730vm-test-run-nixbot-gitlab> gitlab # [  205.983867] systemd[1]: Reached target Multi-User System.1731vm-test-run-nixbot-gitlab> gitlab # [  206.001807] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:06.952Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"bc497f262a01911a83034caa","created_at":"2026-07-04T12:39:04.841Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e762bbfa5ceab4c19004eebc5d6c3256","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.843Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-bc497f262a01911a83034caa: done: 0.023012 sec","job_status":"done","queue_duration_s":2.086674,"scheduling_latency_s":2.08672,"redis_calls":9,"redis_duration_s":0.0027530000000000002,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001509,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001244,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013298,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001129473,"exclusive_lock_hold_duration_s":0.002986692,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.023012,"completed_at":"2026-07-04T12:39:06.952Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001287,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1732vm-test-run-nixbot-gitlab> gitlab # [  206.003778] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:06.954Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"0f33092c01850af81f3137e6","created_at":"2026-07-04T12:39:04.850Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"69218909f77385f1abcd697ef5ce7e31","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.851Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-0f33092c01850af81f3137e6: start","job_status":"start","queue_duration_s":2.102747,"scheduling_latency_s":2.102772}1733vm-test-run-nixbot-gitlab> gitlab # [  206.075687] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:06.971Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"0f33092c01850af81f3137e6","created_at":"2026-07-04T12:39:04.850Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"69218909f77385f1abcd697ef5ce7e31","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.851Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-0f33092c01850af81f3137e6: done: 0.017317 sec","job_status":"done","queue_duration_s":2.102747,"scheduling_latency_s":2.102772,"redis_calls":7,"redis_duration_s":0.0017109999999999998,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001209,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000502,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010118,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017317,"completed_at":"2026-07-04T12:39:06.971Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00081,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1734vm-test-run-nixbot-gitlab> gitlab # [  206.151564] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:06.973Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"43e71f3293aadce1492547cb","created_at":"2026-07-04T12:39:04.856Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ef6365a3f659096f30da7a1bb9e633f9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.859Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-43e71f3293aadce1492547cb: start","job_status":"start","queue_duration_s":2.114471,"scheduling_latency_s":2.114506}1735vm-test-run-nixbot-gitlab> gitlab # [  206.153373] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:06.997Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"43e71f3293aadce1492547cb","created_at":"2026-07-04T12:39:04.856Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ef6365a3f659096f30da7a1bb9e633f9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.859Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-43e71f3293aadce1492547cb: done: 0.024213 sec","job_status":"done","queue_duration_s":2.114471,"scheduling_latency_s":2.114506,"redis_calls":14,"redis_duration_s":0.003332,"redis_read_bytes":16,"redis_write_bytes":4030,"redis_queues_calls":2,"redis_queues_duration_s":0.000354,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002297,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000681,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01694,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024213,"completed_at":"2026-07-04T12:39:06.997Z","load_balancing_strategy":"primary","db_duration_s":0.000891,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1736vm-test-run-nixbot-gitlab> gitlab # [  206.174050] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:06.999Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"f0dec9ae14392f345a1ab239","created_at":"2026-07-04T12:39:04.863Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.865Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-f0dec9ae14392f345a1ab239: start","job_status":"start","queue_duration_s":2.133547,"scheduling_latency_s":2.133572}1737vm-test-run-nixbot-gitlab> gitlab # [  206.196634] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:07.083Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"f0dec9ae14392f345a1ab239","created_at":"2026-07-04T12:39:04.863Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.865Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-f0dec9ae14392f345a1ab239: done: 0.084191 sec","job_status":"done","queue_duration_s":2.133547,"scheduling_latency_s":2.133572,"redis_calls":60,"redis_duration_s":0.015296,"redis_read_bytes":108,"redis_write_bytes":41109,"redis_queues_calls":2,"redis_queues_duration_s":0.000661,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.013915,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00072,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.068402,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.084191,"completed_at":"2026-07-04T12:39:07.083Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1738vm-test-run-nixbot-gitlab> gitlab # [  206.262434] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:07.085Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"8695f9e81c8902fb131c1b97","created_at":"2026-07-04T12:39:04.889Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8494912abf2534cdaee92d41f63f7a6f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-04T12:39:04.895Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-8695f9e81c8902fb131c1b97: start","job_status":"start","queue_duration_s":2.189473,"scheduling_latency_s":2.189501}1739vm-test-run-nixbot-gitlab> gitlab # [  206.341297] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:07.093Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1740vm-test-run-nixbot-gitlab> gitlab # [  206.342825] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:07.097Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"8695f9e81c8902fb131c1b97","created_at":"2026-07-04T12:39:04.889Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8494912abf2534cdaee92d41f63f7a6f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-04T12:39:04.895Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-8695f9e81c8902fb131c1b97: done: 0.012606 sec","job_status":"done","queue_duration_s":2.189473,"scheduling_latency_s":2.189501,"redis_calls":7,"redis_duration_s":0.00163,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001189,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000441,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0072,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012606,"completed_at":"2026-07-04T12:39:07.097Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1741vm-test-run-nixbot-gitlab> gitlab # [  206.351199] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:07.100Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"762cb031e6508fd5828ae527","created_at":"2026-07-04T12:39:04.903Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"19fb7dafae495ffe84942bf616f372d4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.906Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-762cb031e6508fd5828ae527: start","job_status":"start","queue_duration_s":2.193452,"scheduling_latency_s":2.193473}1742vm-test-run-nixbot-gitlab> gitlab # [  206.374420] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:07.107Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1743vm-test-run-nixbot-gitlab> gitlab # [  206.380115] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:07.115Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"762cb031e6508fd5828ae527","created_at":"2026-07-04T12:39:04.903Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"19fb7dafae495ffe84942bf616f372d4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.906Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-762cb031e6508fd5828ae527: done: 0.015917 sec","job_status":"done","queue_duration_s":2.193452,"scheduling_latency_s":2.193473,"redis_calls":7,"redis_duration_s":0.0021780000000000002,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001655,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000523,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009289,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015917,"completed_at":"2026-07-04T12:39:07.115Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1744vm-test-run-nixbot-gitlab> gitlab # [  206.486300] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:07.117Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"1a9610e3bd7276e9a6e0bffe","created_at":"2026-07-04T12:39:04.918Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ac06f925a40655390b9c65aea19ff2f6","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.923Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-1a9610e3bd7276e9a6e0bffe: start","job_status":"start","queue_duration_s":2.19454,"scheduling_latency_s":2.194569}1745vm-test-run-nixbot-gitlab> gitlab # [  206.487814] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:07.139Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"1a9610e3bd7276e9a6e0bffe","created_at":"2026-07-04T12:39:04.918Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ac06f925a40655390b9c65aea19ff2f6","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.923Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-1a9610e3bd7276e9a6e0bffe: done: 0.021723 sec","job_status":"done","queue_duration_s":2.19454,"scheduling_latency_s":2.194569,"redis_calls":10,"redis_duration_s":0.002554,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001474,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00108,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011792,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.006099099,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.021723,"completed_at":"2026-07-04T12:39:07.139Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00444,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1746vm-test-run-nixbot-gitlab> gitlab # [  206.536162] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:07.140Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"e4370cf440a0cdd82bae6171","created_at":"2026-07-04T12:39:04.923Z","trace_propagation_headers":{"sentry-trace":"d311a4d5f6d54ca6a50d968410119c29-d153f669afdb4c15","baggage":"sentry-trace_id=d311a4d5f6d54ca6a50d968410119c29,sentry-release=Unknown"},"correlation_id":"01KWPJ7E1BWTSTSTFPBQNWTB5G","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.925Z","job_size_bytes":23,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-e4370cf440a0cdd82bae6171: start","job_status":"start","queue_duration_s":2.21516,"scheduling_latency_s":2.21518}1747vm-test-run-nixbot-gitlab> gitlab # [  206.561609] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:07.403Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"e4370cf440a0cdd82bae6171","created_at":"2026-07-04T12:39:04.923Z","trace_propagation_headers":{"sentry-trace":"d311a4d5f6d54ca6a50d968410119c29-d153f669afdb4c15","baggage":"sentry-trace_id=d311a4d5f6d54ca6a50d968410119c29,sentry-release=Unknown"},"correlation_id":"01KWPJ7E1BWTSTSTFPBQNWTB5G","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.925Z","job_size_bytes":23,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-e4370cf440a0cdd82bae6171: done: 0.26299 sec","job_status":"done","queue_duration_s":2.21516,"scheduling_latency_s":2.21518,"gitaly_calls":1,"gitaly_duration_s":0.227904,"redis_calls":16,"redis_duration_s":0.004276,"redis_read_bytes":52,"redis_write_bytes":2048,"redis_queues_calls":1,"redis_queues_duration_s":0.00031,"redis_queues_read_bytes":1,"redis_queues_write_bytes":979,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001323,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.001931,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000712,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024101,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.26299,"completed_at":"2026-07-04T12:39:07.403Z","load_balancing_strategy":"primary","db_duration_s":0.005544,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1748vm-test-run-nixbot-gitlab> gitlab # [  206.632893] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:07.405Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"cd7f81d711b80441e1b0faa7","created_at":"2026-07-04T12:39:04.929Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0fd819363a07b1c027a056580c594562","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.930Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-cd7f81d711b80441e1b0faa7: start","job_status":"start","queue_duration_s":2.474382,"scheduling_latency_s":2.474412}1749vm-test-run-nixbot-gitlab> gitlab # [  206.656190] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:07.414Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1750vm-test-run-nixbot-gitlab> gitlab # [  206.661788] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:07.421Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"cd7f81d711b80441e1b0faa7","created_at":"2026-07-04T12:39:04.929Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0fd819363a07b1c027a056580c594562","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.930Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-cd7f81d711b80441e1b0faa7: done: 0.015988 sec","job_status":"done","queue_duration_s":2.474382,"scheduling_latency_s":2.474412,"redis_calls":7,"redis_duration_s":0.00201,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001471,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000539,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010015,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015988,"completed_at":"2026-07-04T12:39:07.421Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1751vm-test-run-nixbot-gitlab> gitlab # [  206.756508] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:07.423Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"80d51019749f3e4d265762eb","created_at":"2026-07-04T12:39:04.939Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3fd58a59b727a0f41a2eef4e24c5365c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.941Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-80d51019749f3e4d265762eb: start","job_status":"start","queue_duration_s":2.481622,"scheduling_latency_s":2.481651}1752vm-test-run-nixbot-gitlab> gitlab # [  206.758932] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:07.452Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"80d51019749f3e4d265762eb","created_at":"2026-07-04T12:39:04.939Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3fd58a59b727a0f41a2eef4e24c5365c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.941Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-80d51019749f3e4d265762eb: done: 0.028792 sec","job_status":"done","queue_duration_s":2.481622,"scheduling_latency_s":2.481651,"redis_calls":9,"redis_duration_s":0.00231,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00126,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00105,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016935,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000522692,"exclusive_lock_hold_duration_s":0.014601017,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.028792,"completed_at":"2026-07-04T12:39:07.452Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006883,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1753vm-test-run-nixbot-gitlab> gitlab # [  206.877391] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:07.453Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"e6f59d20f865a3f2f340febc","created_at":"2026-07-04T12:39:04.944Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f3907175ad6632adc6e42606b0c5183a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.946Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-e6f59d20f865a3f2f340febc: start","job_status":"start","queue_duration_s":2.507808,"scheduling_latency_s":2.50783}1754vm-test-run-nixbot-gitlab> gitlab # [  206.880788] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:07.476Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"e6f59d20f865a3f2f340febc","created_at":"2026-07-04T12:39:04.944Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f3907175ad6632adc6e42606b0c5183a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.946Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-e6f59d20f865a3f2f340febc: done: 0.02238 sec","job_status":"done","queue_duration_s":2.507808,"scheduling_latency_s":2.50783,"redis_calls":9,"redis_duration_s":0.007672999999999999,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003538,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004135,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008904,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000604546,"exclusive_lock_hold_duration_s":0.004060013,"duration_s":0.02238,"completed_at":"2026-07-04T12:39:07.476Z","load_balancing_strategy":"primary","db_duration_s":0.002648,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1755vm-test-run-nixbot-gitlab> gitlab # [  206.903399] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:07.477Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"04c62e3d95b26238c951279b","created_at":"2026-07-04T12:39:04.953Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e8f213b6724c82cc31c8148b7b20f060","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-04T12:39:04.955Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-04c62e3d95b26238c951279b: start","job_status":"start","queue_duration_s":2.522635,"scheduling_latency_s":2.522656}1756vm-test-run-nixbot-gitlab> gitlab # [  206.924909] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:07.494Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"04c62e3d95b26238c951279b","created_at":"2026-07-04T12:39:04.953Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e8f213b6724c82cc31c8148b7b20f060","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-04T12:39:04.955Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-04c62e3d95b26238c951279b: done: 0.017099 sec","job_status":"done","queue_duration_s":2.522635,"scheduling_latency_s":2.522656,"redis_calls":7,"redis_duration_s":0.0017269999999999998,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001241,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000486,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011352,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017099,"completed_at":"2026-07-04T12:39:07.494Z","load_balancing_strategy":"primary","db_duration_s":0.001099,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1757vm-test-run-nixbot-gitlab> gitlab # [  207.038208] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:07.496Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"6dcb305ed7265c06b6b9e5ea","created_at":"2026-07-04T12:39:04.959Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f2e8aebed8f5c10dd163a92b337036fd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.966Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-6dcb305ed7265c06b6b9e5ea: start","job_status":"start","queue_duration_s":2.529672,"scheduling_latency_s":2.529694}1758vm-test-run-nixbot-gitlab> gitlab # [  207.039729] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:07.503Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1759vm-test-run-nixbot-gitlab> gitlab # [  207.044161] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:07.508Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"6dcb305ed7265c06b6b9e5ea","created_at":"2026-07-04T12:39:04.959Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f2e8aebed8f5c10dd163a92b337036fd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.966Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-6dcb305ed7265c06b6b9e5ea: done: 0.012392 sec","job_status":"done","queue_duration_s":2.529672,"scheduling_latency_s":2.529694,"redis_calls":7,"redis_duration_s":0.001704,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001175,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000529,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006959,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012392,"completed_at":"2026-07-04T12:39:07.508Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1760vm-test-run-nixbot-gitlab> gitlab # [  207.082135] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:07.510Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"a5565a5c7a7416a798984408","created_at":"2026-07-04T12:39:04.980Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2b8aba175cc3a47b11b14cac96436d9c","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.987Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-a5565a5c7a7416a798984408: start","job_status":"start","queue_duration_s":2.522625,"scheduling_latency_s":2.522645}1761vm-test-run-nixbot-gitlab> gitlab # [  207.104885] gitaly[1392]: time="2026-07-04T12:39:07.391Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=646 command.oublock=0 command.real_time_ms=221 command.spawn_token_fork_ms=211 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-07-04T12:39:37.167" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:07.167" grpc.time_ms=223.464 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1762vm-test-run-nixbot-gitlab> gitlab # [  207.182669] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:07.527Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"a5565a5c7a7416a798984408","created_at":"2026-07-04T12:39:04.980Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2b8aba175cc3a47b11b14cac96436d9c","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.987Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-a5565a5c7a7416a798984408: done: 0.017313 sec","job_status":"done","queue_duration_s":2.522625,"scheduling_latency_s":2.522645,"redis_calls":7,"redis_duration_s":0.001802,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001323,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000479,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01011,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017313,"completed_at":"2026-07-04T12:39:07.527Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003029,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1763vm-test-run-nixbot-gitlab> gitlab # [  207.190192] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:07.529Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"2062278311fa46df4db2523e","created_at":"2026-07-04T12:39:04.994Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8112c6e96554e1375d2310ecca798006","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.997Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-2062278311fa46df4db2523e: start","job_status":"start","queue_duration_s":2.531508,"scheduling_latency_s":2.531531}1764vm-test-run-nixbot-gitlab> gitlab # [  207.271423] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:07.545Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"2062278311fa46df4db2523e","created_at":"2026-07-04T12:39:04.994Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8112c6e96554e1375d2310ecca798006","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:04.997Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-2062278311fa46df4db2523e: done: 0.016283 sec","job_status":"done","queue_duration_s":2.531508,"scheduling_latency_s":2.531531,"redis_calls":7,"redis_duration_s":0.001654,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001175,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000479,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007642,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.016283,"completed_at":"2026-07-04T12:39:07.545Z","load_balancing_strategy":"primary","db_duration_s":0.004097,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1765vm-test-run-nixbot-gitlab> gitlab # [  207.276278] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:07.546Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"94f1e18ee3bb78f75c571135","created_at":"2026-07-04T12:39:05.015Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bd997bbdcb383c9aa7dddb7b5fd767ae","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-04T12:39:05.022Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-94f1e18ee3bb78f75c571135: start","job_status":"start","queue_duration_s":2.524521,"scheduling_latency_s":2.524544}1766vm-test-run-nixbot-gitlab> gitlab # [  207.345354] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:07.554Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1767vm-test-run-nixbot-gitlab> gitlab # [  207.346873] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:07.558Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"94f1e18ee3bb78f75c571135","created_at":"2026-07-04T12:39:05.015Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7227f41cf8a342ef","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bd997bbdcb383c9aa7dddb7b5fd767ae","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-04T12:39:05.022Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-94f1e18ee3bb78f75c571135: done: 0.011942 sec","job_status":"done","queue_duration_s":2.524521,"scheduling_latency_s":2.524544,"redis_calls":7,"redis_duration_s":0.00152,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001098,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000422,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007629,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011942,"completed_at":"2026-07-04T12:39:07.558Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1768vm-test-run-nixbot-gitlab> gitlab # [  207.365257] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:07.560Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"61bfc49509741a41124107bd","created_at":"2026-07-04T12:39:05.447Z","trace_propagation_headers":{"sentry-trace":"d311a4d5f6d54ca6a50d968410119c29-d153f669afdb4c15","baggage":"sentry-trace_id=d311a4d5f6d54ca6a50d968410119c29,sentry-release=Unknown"},"correlation_id":"01KWPJ7E1BWTSTSTFPBQNWTB5G","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ad1518d5cca61a37ea15e54723375162c42aa019c95e6311439f8a4f2677cc30","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:05.458Z","job_size_bytes":467,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-61bfc49509741a41124107bd: start","job_status":"start","queue_duration_s":2.101996,"scheduling_latency_s":2.102019}1769vm-test-run-nixbot-gitlab> gitlab # [  207.393281] gitaly[1392]: time="2026-07-04T12:39:07.595Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T12:39:37.594" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:07.594" grpc.time_ms=0.967 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1770vm-test-run-nixbot-gitlab> gitlab # [  207.462352] gitaly[1392]: time="2026-07-04T12:39:07.610Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-04T12:39:17.108" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T12:39:07.599" grpc.time_ms=11.422 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1771vm-test-run-nixbot-gitlab> gitlab # [  207.463900] gitaly[1392]: time="2026-07-04T12:39:07.702Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-04T12:39:37.700" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:07.701" grpc.time_ms=1.251 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1772vm-test-run-nixbot-gitlab> gitlab # [  207.465077] gitaly[1392]: time="2026-07-04T12:39:07.772Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=207 command.oublock=0 command.real_time_ms=64 command.spawn_token_fork_ms=63 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-07-04T12:39:37.705" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:07.705" grpc.time_ms=66.522 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1773vm-test-run-nixbot-gitlab> gitlab # [  207.479113] gitaly[1392]: time="2026-07-04T12:39:07.859Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-04T12:39:17.007" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T12:39:07.848" grpc.time_ms=11.549 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1774vm-test-run-nixbot-gitlab> gitlab # [  207.500419] gitaly[1392]: time="2026-07-04T12:39:08.395Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=962 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-04T12:39:38.383" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:08.383" grpc.time_ms=11.439 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1775vm-test-run-nixbot-gitlab> gitlab # [  207.525473] sidekiq-cluster[2150]: {"severity":"WARN","time":"2026-07-04T12:39:08.438Z","class":"CreatePipelineWorker","correlation_id":"01KWPJ7E1BWTSTSTFPBQNWTB5G","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"fd1fc7ecbe895687b4904ab556cca5a4e96ea8fb","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"fd1fc7ecbe895687b4904ab556cca5a4e96ea8fb","ref":"refs/heads/master"},"retry":0}1776vm-test-run-nixbot-gitlab> gitlab # [  207.536750] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.444Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"61bfc49509741a41124107bd","created_at":"2026-07-04T12:39:05.447Z","trace_propagation_headers":{"sentry-trace":"d311a4d5f6d54ca6a50d968410119c29-d153f669afdb4c15","baggage":"sentry-trace_id=d311a4d5f6d54ca6a50d968410119c29,sentry-release=Unknown"},"correlation_id":"01KWPJ7E1BWTSTSTFPBQNWTB5G","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ad1518d5cca61a37ea15e54723375162c42aa019c95e6311439f8a4f2677cc30","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:05.458Z","job_size_bytes":467,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-61bfc49509741a41124107bd: done: 0.884405 sec","job_status":"done","queue_duration_s":2.101996,"scheduling_latency_s":2.102019,"gitaly_calls":7,"gitaly_duration_s":0.133508,"redis_calls":53,"redis_duration_s":0.011468,"redis_read_bytes":171,"redis_write_bytes":4426,"redis_cache_calls":4,"redis_cache_duration_s":0.001028,"redis_cache_read_bytes":10,"redis_cache_write_bytes":342,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.005132,"redis_feature_flag_read_bytes":18,"redis_feature_flag_write_bytes":2433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001492,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002646,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00117,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":40,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":40,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.107,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.102,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.595084,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000396698,"duration_s":0.884405,"completed_at":"2026-07-04T12:39:08.444Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.129781,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1777vm-test-run-nixbot-gitlab> gitlab # [  207.614371] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.446Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"c18fc382d77b77ce7e3e4e7f","created_at":"2026-07-04T12:39:05.695Z","trace_propagation_headers":{"sentry-trace":"d311a4d5f6d54ca6a50d968410119c29-b41d213aded34fe7","baggage":"sentry-trace_id=d311a4d5f6d54ca6a50d968410119c29,sentry-release=Unknown"},"correlation_id":"01KWPJ7E1BWTSTSTFPBQNWTB5G","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:05.700Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-c18fc382d77b77ce7e3e4e7f: start","job_status":"start","queue_duration_s":2.745895,"scheduling_latency_s":2.745917}1778vm-test-run-nixbot-gitlab> gitlab # [  207.640293] gitaly[1392]: time="2026-07-04T12:39:08.410Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7E1BWTSTSTFPBQNWTB5G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-07-04T12:39:38.399" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:08.399" grpc.time_ms=10.401 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1779vm-test-run-nixbot-gitlab> gitlab # [  207.720487] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.483Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"c18fc382d77b77ce7e3e4e7f","created_at":"2026-07-04T12:39:05.695Z","trace_propagation_headers":{"sentry-trace":"d311a4d5f6d54ca6a50d968410119c29-b41d213aded34fe7","baggage":"sentry-trace_id=d311a4d5f6d54ca6a50d968410119c29,sentry-release=Unknown"},"correlation_id":"01KWPJ7E1BWTSTSTFPBQNWTB5G","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:05.700Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-c18fc382d77b77ce7e3e4e7f: done: 0.037288 sec","job_status":"done","queue_duration_s":2.745895,"scheduling_latency_s":2.745917,"redis_calls":19,"redis_duration_s":0.004512,"redis_read_bytes":22,"redis_write_bytes":4289,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001087,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000529,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2260,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002107,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000789,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024287,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.006487697000000001,"duration_s":0.037288,"completed_at":"2026-07-04T12:39:08.483Z","load_balancing_strategy":"primary","db_duration_s":0.005358,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1780vm-test-run-nixbot-gitlab> gitlab # [  207.792270] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.484Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4f992d8777879d059582927e","created_at":"2026-07-04T12:39:06.985Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7acbd3c97c604f82","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ef6365a3f659096f30da7a1bb9e633f9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:06.992Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4f992d8777879d059582927e: start","job_status":"start","queue_duration_s":1.49195,"scheduling_latency_s":1.491971}1781vm-test-run-nixbot-gitlab> gitlab # [  207.793766] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.495Z","message":"No existing merge request to be cleaned up."}1782vm-test-run-nixbot-gitlab> gitlab # [  207.794897] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.501Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4f992d8777879d059582927e","created_at":"2026-07-04T12:39:06.985Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7acbd3c97c604f82","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ef6365a3f659096f30da7a1bb9e633f9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:06.992Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4f992d8777879d059582927e: done: 0.016787 sec","job_status":"done","queue_duration_s":1.49195,"scheduling_latency_s":1.491971,"redis_calls":10,"redis_duration_s":0.0026160000000000003,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000981,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001635,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010508,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016787,"completed_at":"2026-07-04T12:39:08.501Z","load_balancing_strategy":"primary","db_duration_s":0.001559,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1783vm-test-run-nixbot-gitlab> gitlab # [  207.815301] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.503Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"58ccc7aced8f5640faa3d1c3","created_at":"2026-07-04T12:39:06.985Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7acbd3c97c604f82","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ef6365a3f659096f30da7a1bb9e633f9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:06.992Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-58ccc7aced8f5640faa3d1c3: start","job_status":"start","queue_duration_s":1.510093,"scheduling_latency_s":1.510114}1784vm-test-run-nixbot-gitlab> gitlab # [  207.835170] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.513Z","message":"No existing merge request to be cleaned up."}1785vm-test-run-nixbot-gitlab> gitlab # [  207.903740] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.525Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"58ccc7aced8f5640faa3d1c3","created_at":"2026-07-04T12:39:06.985Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7acbd3c97c604f82","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ef6365a3f659096f30da7a1bb9e633f9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:06.992Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-58ccc7aced8f5640faa3d1c3: done: 0.022125 sec","job_status":"done","queue_duration_s":1.510093,"scheduling_latency_s":1.510114,"redis_calls":10,"redis_duration_s":0.003568,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001245,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002323,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013097,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022125,"completed_at":"2026-07-04T12:39:08.525Z","load_balancing_strategy":"primary","db_duration_s":0.003581,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1786vm-test-run-nixbot-gitlab> gitlab # [  207.906095] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.526Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"193f14cede3f9d333e6abb6d","created_at":"2026-07-04T12:39:06.985Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7acbd3c97c604f82","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ef6365a3f659096f30da7a1bb9e633f9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:06.992Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-193f14cede3f9d333e6abb6d: start","job_status":"start","queue_duration_s":1.533682,"scheduling_latency_s":1.53371}1787vm-test-run-nixbot-gitlab> gitlab # [  207.917766] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.537Z","message":"No existing merge request to be cleaned up."}1788vm-test-run-nixbot-gitlab> gitlab # [  207.920436] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.543Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"193f14cede3f9d333e6abb6d","created_at":"2026-07-04T12:39:06.985Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7acbd3c97c604f82","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ef6365a3f659096f30da7a1bb9e633f9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:06.992Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-193f14cede3f9d333e6abb6d: done: 0.017215 sec","job_status":"done","queue_duration_s":1.533682,"scheduling_latency_s":1.53371,"redis_calls":10,"redis_duration_s":0.002462,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00096,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001502,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010874,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017215,"completed_at":"2026-07-04T12:39:08.543Z","load_balancing_strategy":"primary","db_duration_s":0.001636,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1789vm-test-run-nixbot-gitlab> gitlab # [  207.982389] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.545Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"627e28eb840a613e5e69c714","created_at":"2026-07-04T12:39:06.985Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7acbd3c97c604f82","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ef6365a3f659096f30da7a1bb9e633f9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:06.992Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-627e28eb840a613e5e69c714: start","job_status":"start","queue_duration_s":1.552418,"scheduling_latency_s":1.552442}1790vm-test-run-nixbot-gitlab> gitlab # [  208.002290] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.555Z","message":"No existing merge request to be cleaned up."}1791vm-test-run-nixbot-gitlab> gitlab # [  208.005371] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.561Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"627e28eb840a613e5e69c714","created_at":"2026-07-04T12:39:06.985Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-7acbd3c97c604f82","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ef6365a3f659096f30da7a1bb9e633f9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:06.992Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-627e28eb840a613e5e69c714: done: 0.016377 sec","job_status":"done","queue_duration_s":1.552418,"scheduling_latency_s":1.552442,"redis_calls":10,"redis_duration_s":0.002453,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00096,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001493,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010084,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016377,"completed_at":"2026-07-04T12:39:08.561Z","load_balancing_strategy":"primary","db_duration_s":0.001856,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1792vm-test-run-nixbot-gitlab> gitlab # [  208.077953] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.563Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"745afe2100e556183b9fa23f","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-745afe2100e556183b9fa23f: start","job_status":"start","queue_duration_s":1.487704,"scheduling_latency_s":1.487745}1793vm-test-run-nixbot-gitlab> gitlab # [  208.084330] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.582Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"745afe2100e556183b9fa23f","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-745afe2100e556183b9fa23f: done: 0.018627 sec","job_status":"done","queue_duration_s":1.487704,"scheduling_latency_s":1.487745,"redis_calls":13,"redis_duration_s":0.002975,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000929,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002046,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01103,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018627,"completed_at":"2026-07-04T12:39:08.582Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1794vm-test-run-nixbot-gitlab> gitlab # [  208.158041] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.583Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b6fe135162099e22a2413fe","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b6fe135162099e22a2413fe: start","job_status":"start","queue_duration_s":1.507524,"scheduling_latency_s":1.507546}1795vm-test-run-nixbot-gitlab> gitlab # [  208.165788] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.598Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b6fe135162099e22a2413fe","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b6fe135162099e22a2413fe: done: 0.01529 sec","job_status":"done","queue_duration_s":1.507524,"scheduling_latency_s":1.507546,"redis_calls":13,"redis_duration_s":0.00309,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001038,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002052,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009559,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01529,"completed_at":"2026-07-04T12:39:08.598Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1796vm-test-run-nixbot-gitlab> gitlab # [  208.229280] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.600Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b41f0a3d4e4312a135c17bfa","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b41f0a3d4e4312a135c17bfa: start","job_status":"start","queue_duration_s":1.524195,"scheduling_latency_s":1.524216}1797vm-test-run-nixbot-gitlab> gitlab # [  208.249109] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.616Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b41f0a3d4e4312a135c17bfa","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b41f0a3d4e4312a135c17bfa: done: 0.015733 sec","job_status":"done","queue_duration_s":1.524195,"scheduling_latency_s":1.524216,"redis_calls":13,"redis_duration_s":0.003081,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000922,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002159,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01008,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015733,"completed_at":"2026-07-04T12:39:08.616Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1798vm-test-run-nixbot-gitlab> gitlab # [  208.367296] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.617Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4228fef2ad4aca0a76c7222e","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4228fef2ad4aca0a76c7222e: start","job_status":"start","queue_duration_s":1.541205,"scheduling_latency_s":1.541226}1799vm-test-run-nixbot-gitlab> gitlab # [  208.368796] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.634Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4228fef2ad4aca0a76c7222e","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4228fef2ad4aca0a76c7222e: done: 0.017129 sec","job_status":"done","queue_duration_s":1.541205,"scheduling_latency_s":1.541226,"redis_calls":13,"redis_duration_s":0.00332,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001192,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002128,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010399,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017129,"completed_at":"2026-07-04T12:39:08.634Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1800vm-test-run-nixbot-gitlab> gitlab # [  208.393171] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.635Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"043f25ca2a47340b659ce4d6","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-043f25ca2a47340b659ce4d6: start","job_status":"start","queue_duration_s":1.559756,"scheduling_latency_s":1.559778}1801vm-test-run-nixbot-gitlab> gitlab # [  208.413148] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.651Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"043f25ca2a47340b659ce4d6","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-043f25ca2a47340b659ce4d6: done: 0.015654 sec","job_status":"done","queue_duration_s":1.559756,"scheduling_latency_s":1.559778,"redis_calls":13,"redis_duration_s":0.003011,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001034,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001977,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010544,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015654,"completed_at":"2026-07-04T12:39:08.651Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1802vm-test-run-nixbot-gitlab> gitlab # [  208.513044] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.652Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"364248f507ccc75183d608df","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-364248f507ccc75183d608df: start","job_status":"start","queue_duration_s":1.576741,"scheduling_latency_s":1.576762}1803vm-test-run-nixbot-gitlab> gitlab # [  208.514870] gitaly[1392]: time="2026-07-04T12:39:08.631Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=962 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7HDMDRZ43F42EENHP8G2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-04T12:39:38.017" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:08.618" grpc.time_ms=13.027 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1804vm-test-run-nixbot-gitlab> gitlab # [  208.519788] nixbot[3144]: {"ts": "2026-07-04T12:39:09.039Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1805vm-test-run-nixbot-gitlab> gitlab # [  208.523616] nixbot[3144]: {"ts": "2026-07-04T12:39:09.109Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1806vm-test-run-nixbot-gitlab> gitlab # [  208.527106] nixbot[3144]: {"ts": "2026-07-04T12:39:09.176Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1807vm-test-run-nixbot-gitlab> gitlab # [  208.532225] nixbot[3144]: {"ts": "2026-07-04T12:39:09.177Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1808vm-test-run-nixbot-gitlab> gitlab # [  208.534223] nixbot[3144]: {"ts": "2026-07-04T12:39:09.292Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""}1809vm-test-run-nixbot-gitlab> gitlab # [  208.538572] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:09 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 21051810vm-test-run-nixbot-gitlab> gitlab # [  208.541817] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:09 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 571811vm-test-run-nixbot-gitlab> gitlab # [  208.544967] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:09 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 868 "" "python-httpx/0.28.1" 1101812vm-test-run-nixbot-gitlab> gitlab # [  208.548141] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:09 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 1811813vm-test-run-nixbot-gitlab> gitlab # [  208.552310] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.671Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"364248f507ccc75183d608df","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-364248f507ccc75183d608df: done: 0.018156 sec","job_status":"done","queue_duration_s":1.576741,"scheduling_latency_s":1.576762,"redis_calls":13,"redis_duration_s":0.003399,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000915,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002484,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012406,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018156,"completed_at":"2026-07-04T12:39:08.671Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1814vm-test-run-nixbot-gitlab> gitlab # [  208.651977] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.672Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee54746d2f7172088d5a2839","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee54746d2f7172088d5a2839: start","job_status":"start","queue_duration_s":1.596634,"scheduling_latency_s":1.596663}1815vm-test-run-nixbot-gitlab> gitlab # [  208.654138] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.688Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee54746d2f7172088d5a2839","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee54746d2f7172088d5a2839: done: 0.016152 sec","job_status":"done","queue_duration_s":1.596634,"scheduling_latency_s":1.596663,"redis_calls":13,"redis_duration_s":0.0028899999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000869,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002021,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010418,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016152,"completed_at":"2026-07-04T12:39:08.688Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1816vm-test-run-nixbot-gitlab> gitlab # [  208.722395] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.690Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8971204b1551f16b70415d3","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8971204b1551f16b70415d3: start","job_status":"start","queue_duration_s":1.613983,"scheduling_latency_s":1.614004}1817vm-test-run-nixbot-gitlab> gitlab # [  208.723455] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.705Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8971204b1551f16b70415d3","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8971204b1551f16b70415d3: done: 0.015263 sec","job_status":"done","queue_duration_s":1.613983,"scheduling_latency_s":1.614004,"redis_calls":13,"redis_duration_s":0.002915,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00089,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002025,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010082,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015263,"completed_at":"2026-07-04T12:39:08.705Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1818vm-test-run-nixbot-gitlab> gitlab # [  208.800144] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.706Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68ca8b199bf9d50fd27024f4","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68ca8b199bf9d50fd27024f4: start","job_status":"start","queue_duration_s":1.630551,"scheduling_latency_s":1.630573}1819vm-test-run-nixbot-gitlab> gitlab # [  208.801394] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.721Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68ca8b199bf9d50fd27024f4","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68ca8b199bf9d50fd27024f4: done: 0.014998 sec","job_status":"done","queue_duration_s":1.630551,"scheduling_latency_s":1.630573,"redis_calls":13,"redis_duration_s":0.002908,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00086,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002048,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009911,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014998,"completed_at":"2026-07-04T12:39:08.721Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1820vm-test-run-nixbot-gitlab> gitlab # [  208.906740] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.798Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd0c2e62db5e6fb1a5d68720","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd0c2e62db5e6fb1a5d68720: start","job_status":"start","queue_duration_s":1.722198,"scheduling_latency_s":1.722233}1821vm-test-run-nixbot-gitlab> gitlab # [  208.908136] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.816Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd0c2e62db5e6fb1a5d68720","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd0c2e62db5e6fb1a5d68720: done: 0.017703 sec","job_status":"done","queue_duration_s":1.722198,"scheduling_latency_s":1.722233,"redis_calls":13,"redis_duration_s":0.002949,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000941,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002008,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011226,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017703,"completed_at":"2026-07-04T12:39:08.815Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1822vm-test-run-nixbot-gitlab> gitlab # [  208.978126] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.817Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"611c849550daaf895ad47826","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-611c849550daaf895ad47826: start","job_status":"start","queue_duration_s":1.741151,"scheduling_latency_s":1.74118}1823vm-test-run-nixbot-gitlab> gitlab # [  208.980370] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.832Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"611c849550daaf895ad47826","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-611c849550daaf895ad47826: done: 0.015484 sec","job_status":"done","queue_duration_s":1.741151,"scheduling_latency_s":1.74118,"redis_calls":13,"redis_duration_s":0.002912,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000903,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002009,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010385,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015484,"completed_at":"2026-07-04T12:39:08.832Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1824vm-test-run-nixbot-gitlab> gitlab # [  209.089915] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.834Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7fda88065abd93ddc9b020b6","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7fda88065abd93ddc9b020b6: start","job_status":"start","queue_duration_s":1.757922,"scheduling_latency_s":1.757942}1825vm-test-run-nixbot-gitlab> gitlab # [  209.091207] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.849Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7fda88065abd93ddc9b020b6","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7fda88065abd93ddc9b020b6: done: 0.015094 sec","job_status":"done","queue_duration_s":1.757922,"scheduling_latency_s":1.757942,"redis_calls":13,"redis_duration_s":0.002898,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000886,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002012,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010175,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015094,"completed_at":"2026-07-04T12:39:08.849Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1826vm-test-run-nixbot-gitlab> gitlab # [  209.108978] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.850Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dcba77b8dc86380e6792971a","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dcba77b8dc86380e6792971a: start","job_status":"start","queue_duration_s":1.774305,"scheduling_latency_s":1.774325}1827vm-test-run-nixbot-gitlab> gitlab # [  209.124821] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.866Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dcba77b8dc86380e6792971a","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dcba77b8dc86380e6792971a: done: 0.0162 sec","job_status":"done","queue_duration_s":1.774305,"scheduling_latency_s":1.774325,"redis_calls":13,"redis_duration_s":0.003532,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000915,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002617,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009504,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0162,"completed_at":"2026-07-04T12:39:08.866Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1828vm-test-run-nixbot-gitlab> gitlab # [  209.235444] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.868Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"172a1d8a14211bb777aae318","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-172a1d8a14211bb777aae318: start","job_status":"start","queue_duration_s":1.791908,"scheduling_latency_s":1.791929}1829vm-test-run-nixbot-gitlab> gitlab # [  209.236694] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.883Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"172a1d8a14211bb777aae318","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-172a1d8a14211bb777aae318: done: 0.015438 sec","job_status":"done","queue_duration_s":1.791908,"scheduling_latency_s":1.791929,"redis_calls":13,"redis_duration_s":0.003032,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000987,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002045,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009835,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015438,"completed_at":"2026-07-04T12:39:08.883Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1830vm-test-run-nixbot-gitlab> gitlab # [  209.270122] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.884Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b906ca49615215e079ab6cd8","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b906ca49615215e079ab6cd8: start","job_status":"start","queue_duration_s":1.808686,"scheduling_latency_s":1.808707}1831vm-test-run-nixbot-gitlab> gitlab # [  209.290124] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.900Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b906ca49615215e079ab6cd8","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b906ca49615215e079ab6cd8: done: 0.015589 sec","job_status":"done","queue_duration_s":1.808686,"scheduling_latency_s":1.808707,"redis_calls":13,"redis_duration_s":0.003132,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000882,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00225,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010055,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015589,"completed_at":"2026-07-04T12:39:08.900Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1832vm-test-run-nixbot-gitlab> gitlab # [  209.381266] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.901Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad54a49eefb77e4c21fe8d06","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad54a49eefb77e4c21fe8d06: start","job_status":"start","queue_duration_s":1.825567,"scheduling_latency_s":1.82563}1833vm-test-run-nixbot-gitlab> gitlab # [  209.382906] nixbot[3144]: {"ts": "2026-07-04T12:39:09.483Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1834vm-test-run-nixbot-gitlab> gitlab # [  209.383648] nixbot[3144]: {"ts": "2026-07-04T12:39:09.541Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1835vm-test-run-nixbot-gitlab> gitlab # [  209.384890] nixbot[3144]: {"ts": "2026-07-04T12:39:09.605Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""}1836vm-test-run-nixbot-gitlab> gitlab # [  209.386068] nixbot[3144]: {"ts": "2026-07-04T12:39:09.718Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""}1837vm-test-run-nixbot-gitlab> gitlab # [  209.386905] nixbot[3144]: {"ts": "2026-07-04T12:39:09.775Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""}1838vm-test-run-nixbot-gitlab> gitlab # [  209.390332] nixbot[3144]: {"ts": "2026-07-04T12:39:09.778Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1839vm-test-run-nixbot-gitlab> gitlab # [  209.394419] gitaly[1392]: time="2026-07-04T12:39:09.660Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7M1B9ZGTCBRMH7GA6F43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-07-04T12:39:39.049" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T12:39:09.650" grpc.time_ms=9.692 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1840vm-test-run-nixbot-gitlab> gitlab # [  209.466108] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:09 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "python-httpx/0.28.1" 481841vm-test-run-nixbot-gitlab> gitlab # [  209.467330] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:09 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 868 "" "python-httpx/0.28.1" 591842vm-test-run-nixbot-gitlab> gitlab # [  209.468222] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:09 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 731 "" "python-httpx/0.28.1" 1051843vm-test-run-nixbot-gitlab> gitlab # [  209.469072] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:09 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 521844vm-test-run-nixbot-gitlab> gitlab # [  209.469906] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:09 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 571845vm-test-run-nixbot-gitlab> gitlab # [  209.471097] gitlab-workhorse[1293]: time="2026-07-04T12:39:09Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1491846vm-test-run-nixbot-gitlab> gitlab # [  209.471907] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:09 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 149 "" "git/2.54.0" 731847vm-test-run-nixbot-gitlab> gitlab # [  209.473106] gitlab-workhorse[1293]: time="2026-07-04T12:39:10Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3731848vm-test-run-nixbot-gitlab> gitlab # [  209.473936] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:10 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 373 "" "git/2.54.0" 641849vm-test-run-nixbot-gitlab> gitlab # [  209.475098] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:10 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 591850vm-test-run-nixbot-gitlab> gitlab # [  209.475662] gitlab-workhorse[1293]: time="2026-07-04T12:39:10Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671851vm-test-run-nixbot-gitlab> gitlab # [  209.476811] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:10 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 611852vm-test-run-nixbot-gitlab> gitlab # [  209.478078] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.917Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad54a49eefb77e4c21fe8d06","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad54a49eefb77e4c21fe8d06: done: 0.015806 sec","job_status":"done","queue_duration_s":1.825567,"scheduling_latency_s":1.82563,"redis_calls":13,"redis_duration_s":0.003277,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001219,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002058,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009885,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015806,"completed_at":"2026-07-04T12:39:08.917Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1853vm-test-run-nixbot-gitlab> gitlab # [  209.568209] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.919Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"763e377fd142822fa5db80d5","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-763e377fd142822fa5db80d5: start","job_status":"start","queue_duration_s":1.842863,"scheduling_latency_s":1.842885}1854vm-test-run-nixbot-gitlab> gitlab # [  209.569723] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.934Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"763e377fd142822fa5db80d5","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-763e377fd142822fa5db80d5: done: 0.015836 sec","job_status":"done","queue_duration_s":1.842863,"scheduling_latency_s":1.842885,"redis_calls":13,"redis_duration_s":0.0030009999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000957,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002044,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009447,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015836,"completed_at":"2026-07-04T12:39:08.934Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1855vm-test-run-nixbot-gitlab> gitlab # [  209.663251] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.936Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c24456addae8e2d75e99cb55","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c24456addae8e2d75e99cb55: start","job_status":"start","queue_duration_s":1.860055,"scheduling_latency_s":1.860076}1856vm-test-run-nixbot-gitlab> gitlab # [  209.664857] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.951Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c24456addae8e2d75e99cb55","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c24456addae8e2d75e99cb55: done: 0.015354 sec","job_status":"done","queue_duration_s":1.860055,"scheduling_latency_s":1.860076,"redis_calls":13,"redis_duration_s":0.00296,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000976,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001984,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009799,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015354,"completed_at":"2026-07-04T12:39:08.951Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1857vm-test-run-nixbot-gitlab> gitlab # [  209.684254] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.952Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8034486525c1fb7b08e9f0a5","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8034486525c1fb7b08e9f0a5: start","job_status":"start","queue_duration_s":1.876728,"scheduling_latency_s":1.876749}1858vm-test-run-nixbot-gitlab> gitlab # [  209.704229] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.968Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8034486525c1fb7b08e9f0a5","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8034486525c1fb7b08e9f0a5: done: 0.015813 sec","job_status":"done","queue_duration_s":1.876728,"scheduling_latency_s":1.876749,"redis_calls":13,"redis_duration_s":0.003123,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000934,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002189,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010208,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015813,"completed_at":"2026-07-04T12:39:08.968Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1859vm-test-run-nixbot-gitlab> gitlab # [  209.798057] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.970Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"14cbf4a74c3320f07876b56c","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-14cbf4a74c3320f07876b56c: start","job_status":"start","queue_duration_s":1.893878,"scheduling_latency_s":1.8939}1860vm-test-run-nixbot-gitlab> gitlab # [  209.799343] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.985Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"14cbf4a74c3320f07876b56c","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-14cbf4a74c3320f07876b56c: done: 0.015158 sec","job_status":"done","queue_duration_s":1.893878,"scheduling_latency_s":1.8939,"redis_calls":13,"redis_duration_s":0.002915,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000901,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002014,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010143,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015158,"completed_at":"2026-07-04T12:39:08.985Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1861vm-test-run-nixbot-gitlab> gitlab # [  209.912510] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:08.986Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dad8316fbc50e1bc9b67efe2","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dad8316fbc50e1bc9b67efe2: start","job_status":"start","queue_duration_s":1.910324,"scheduling_latency_s":1.910345}1862vm-test-run-nixbot-gitlab> gitlab # [  209.913822] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.002Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dad8316fbc50e1bc9b67efe2","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dad8316fbc50e1bc9b67efe2: done: 0.015587 sec","job_status":"done","queue_duration_s":1.910324,"scheduling_latency_s":1.910345,"redis_calls":13,"redis_duration_s":0.002852,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000896,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001956,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009858,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015587,"completed_at":"2026-07-04T12:39:09.002Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1863vm-test-run-nixbot-gitlab> gitlab # [  209.929220] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.003Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e4617d86acee2e809b31820","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e4617d86acee2e809b31820: start","job_status":"start","queue_duration_s":1.927234,"scheduling_latency_s":1.927255}1864vm-test-run-nixbot-gitlab> gitlab # [  209.949249] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.021Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e4617d86acee2e809b31820","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e4617d86acee2e809b31820: done: 0.018097 sec","job_status":"done","queue_duration_s":1.927234,"scheduling_latency_s":1.927255,"redis_calls":13,"redis_duration_s":0.003173,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000981,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002192,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012612,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018097,"completed_at":"2026-07-04T12:39:09.021Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1865vm-test-run-nixbot-gitlab> gitlab # [  210.012508] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.023Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a07e964af6bd8d9b493e36c9","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a07e964af6bd8d9b493e36c9: start","job_status":"start","queue_duration_s":1.946852,"scheduling_latency_s":1.94688}1866vm-test-run-nixbot-gitlab> gitlab # [  210.032372] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.047Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a07e964af6bd8d9b493e36c9","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a07e964af6bd8d9b493e36c9: done: 0.024653 sec","job_status":"done","queue_duration_s":1.946852,"scheduling_latency_s":1.94688,"redis_calls":13,"redis_duration_s":0.008331,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00507,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003261,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013388,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024653,"completed_at":"2026-07-04T12:39:09.047Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1867vm-test-run-nixbot-gitlab> gitlab # [  210.103815] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.049Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1cfac3630a25a8e904c3c032","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1cfac3630a25a8e904c3c032: start","job_status":"start","queue_duration_s":1.972945,"scheduling_latency_s":1.972969}1868vm-test-run-nixbot-gitlab> gitlab # [  210.113217] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.065Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1cfac3630a25a8e904c3c032","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1cfac3630a25a8e904c3c032: done: 0.016384 sec","job_status":"done","queue_duration_s":1.972945,"scheduling_latency_s":1.972969,"redis_calls":13,"redis_duration_s":0.003175,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001045,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00213,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010773,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016384,"completed_at":"2026-07-04T12:39:09.065Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1869vm-test-run-nixbot-gitlab> gitlab # [  210.215417] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.066Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf9adc8c20446842e7e9a1d0","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf9adc8c20446842e7e9a1d0: start","job_status":"start","queue_duration_s":1.990575,"scheduling_latency_s":1.990639}1870vm-test-run-nixbot-gitlab> gitlab # [  210.216718] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.082Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf9adc8c20446842e7e9a1d0","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf9adc8c20446842e7e9a1d0: done: 0.015381 sec","job_status":"done","queue_duration_s":1.990575,"scheduling_latency_s":1.990639,"redis_calls":13,"redis_duration_s":0.0028919999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000887,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002005,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009911,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015381,"completed_at":"2026-07-04T12:39:09.082Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1871vm-test-run-nixbot-gitlab> gitlab # [  210.290161] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.083Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d39babd64d9d7e88a76dedee","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d39babd64d9d7e88a76dedee: start","job_status":"start","queue_duration_s":2.007653,"scheduling_latency_s":2.007674}1872vm-test-run-nixbot-gitlab> gitlab # [  210.292327] gitaly[1392]: time="2026-07-04T12:39:09.671Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7M1B9ZGTCBRMH7GA6F43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T12:39:39.070" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:09.671" grpc.time_ms=0.552 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1873vm-test-run-nixbot-gitlab> gitlab # [  210.295485] nixbot[3144]: {"ts": "2026-07-04T12:39:10.611Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/fd1fc7ecbe895687b4904ab556cca5a4e96ea8fb \"HTTP/1.1 201 Created\""}1874vm-test-run-nixbot-gitlab> gitlab # [  210.299428] nixbot[3144]: {"ts": "2026-07-04T12:39:10.622Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-fd1fc7ecbe89-b613b907", "argv0": "bwrap"}1875vm-test-run-nixbot-gitlab> gitlab # [  210.303515] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.099Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d39babd64d9d7e88a76dedee","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d39babd64d9d7e88a76dedee: done: 0.015563 sec","job_status":"done","queue_duration_s":2.007653,"scheduling_latency_s":2.007674,"redis_calls":13,"redis_duration_s":0.002873,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000869,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002004,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010519,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015563,"completed_at":"2026-07-04T12:39:09.099Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1876vm-test-run-nixbot-gitlab> gitlab # [  210.377192] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.100Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a4346f81919dfce8a22e3ae6","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a4346f81919dfce8a22e3ae6: start","job_status":"start","queue_duration_s":2.024415,"scheduling_latency_s":2.024435}1877vm-test-run-nixbot-gitlab> gitlab # [  210.384180] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.126Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a4346f81919dfce8a22e3ae6","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a4346f81919dfce8a22e3ae6: done: 0.02575 sec","job_status":"done","queue_duration_s":2.024415,"scheduling_latency_s":2.024435,"redis_calls":13,"redis_duration_s":0.009285999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.006307,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002979,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013379,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02575,"completed_at":"2026-07-04T12:39:09.126Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1878vm-test-run-nixbot-gitlab> gitlab # [  210.493901] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.127Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34ba4647243de55af4d1bb73","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34ba4647243de55af4d1bb73: start","job_status":"start","queue_duration_s":2.051525,"scheduling_latency_s":2.051547}1879vm-test-run-nixbot-gitlab> gitlab # [  210.495170] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.145Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34ba4647243de55af4d1bb73","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34ba4647243de55af4d1bb73: done: 0.017576 sec","job_status":"done","queue_duration_s":2.051525,"scheduling_latency_s":2.051547,"redis_calls":13,"redis_duration_s":0.003822,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001678,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002144,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010098,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017576,"completed_at":"2026-07-04T12:39:09.145Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1880vm-test-run-nixbot-gitlab> gitlab # [  210.586957] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.146Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98dc2ed9b930e882147b0585","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98dc2ed9b930e882147b0585: start","job_status":"start","queue_duration_s":2.070353,"scheduling_latency_s":2.070374}1881vm-test-run-nixbot-gitlab> gitlab # [  210.587747] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.167Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98dc2ed9b930e882147b0585","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98dc2ed9b930e882147b0585: done: 0.021186 sec","job_status":"done","queue_duration_s":2.070353,"scheduling_latency_s":2.070374,"redis_calls":13,"redis_duration_s":0.007901,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002253,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005648,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010886,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021186,"completed_at":"2026-07-04T12:39:09.167Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1882vm-test-run-nixbot-gitlab> gitlab # [  210.672826] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.169Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"05bcc6602528287262bb3f35","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-05bcc6602528287262bb3f35: start","job_status":"start","queue_duration_s":2.093012,"scheduling_latency_s":2.093034}1883vm-test-run-nixbot-gitlab> gitlab # [  210.674135] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.191Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"05bcc6602528287262bb3f35","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-05bcc6602528287262bb3f35: done: 0.022311 sec","job_status":"done","queue_duration_s":2.093012,"scheduling_latency_s":2.093034,"redis_calls":13,"redis_duration_s":0.003321,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001014,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002307,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012797,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022311,"completed_at":"2026-07-04T12:39:09.191Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1884vm-test-run-nixbot-gitlab> gitlab # [  210.743847] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.193Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a67f657e9ebc00217ab40683","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a67f657e9ebc00217ab40683: start","job_status":"start","queue_duration_s":2.116885,"scheduling_latency_s":2.116913}1885vm-test-run-nixbot-gitlab> gitlab # [  210.745260] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.209Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a67f657e9ebc00217ab40683","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a67f657e9ebc00217ab40683: done: 0.016555 sec","job_status":"done","queue_duration_s":2.116885,"scheduling_latency_s":2.116913,"redis_calls":13,"redis_duration_s":0.003028,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000984,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002044,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010461,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016555,"completed_at":"2026-07-04T12:39:09.209Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1886vm-test-run-nixbot-gitlab> gitlab # [  210.817190] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.211Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d2af637f29dcc26f884f436f","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d2af637f29dcc26f884f436f: start","job_status":"start","queue_duration_s":2.135297,"scheduling_latency_s":2.135319}1887vm-test-run-nixbot-gitlab> gitlab # [  210.818738] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.227Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d2af637f29dcc26f884f436f","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d2af637f29dcc26f884f436f: done: 0.015645 sec","job_status":"done","queue_duration_s":2.135297,"scheduling_latency_s":2.135319,"redis_calls":13,"redis_duration_s":0.003081,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000891,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00219,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010078,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015645,"completed_at":"2026-07-04T12:39:09.227Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1888vm-test-run-nixbot-gitlab> gitlab # [  210.911957] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.228Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"154d6124ffe1ecc3570c3a6b","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-154d6124ffe1ecc3570c3a6b: start","job_status":"start","queue_duration_s":2.152218,"scheduling_latency_s":2.152239}1889vm-test-run-nixbot-gitlab> gitlab # [  210.913651] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.244Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"154d6124ffe1ecc3570c3a6b","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-154d6124ffe1ecc3570c3a6b: done: 0.015825 sec","job_status":"done","queue_duration_s":2.152218,"scheduling_latency_s":2.152239,"redis_calls":13,"redis_duration_s":0.0030169999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000894,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002123,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010094,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015825,"completed_at":"2026-07-04T12:39:09.244Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1890vm-test-run-nixbot-gitlab> gitlab # [  210.931249] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.245Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"221416b74a325d4f38da33b3","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-221416b74a325d4f38da33b3: start","job_status":"start","queue_duration_s":2.16943,"scheduling_latency_s":2.169452}1891vm-test-run-nixbot-gitlab> gitlab # [  210.951260] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.261Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"221416b74a325d4f38da33b3","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-221416b74a325d4f38da33b3: done: 0.015503 sec","job_status":"done","queue_duration_s":2.16943,"scheduling_latency_s":2.169452,"redis_calls":13,"redis_duration_s":0.003115,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002063,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010179,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015503,"completed_at":"2026-07-04T12:39:09.261Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1892vm-test-run-nixbot-gitlab> gitlab # [  211.050211] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.262Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"154c51cd6a493f3ba75eef80","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-154c51cd6a493f3ba75eef80: start","job_status":"start","queue_duration_s":2.186235,"scheduling_latency_s":2.186256}1893vm-test-run-nixbot-gitlab> gitlab # [  211.051739] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.280Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"154c51cd6a493f3ba75eef80","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-154c51cd6a493f3ba75eef80: done: 0.018396 sec","job_status":"done","queue_duration_s":2.186235,"scheduling_latency_s":2.186256,"redis_calls":13,"redis_duration_s":0.003187,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001003,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002184,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012265,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018396,"completed_at":"2026-07-04T12:39:09.280Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1894vm-test-run-nixbot-gitlab> gitlab # [  211.095252] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.282Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3bb2937ccccc737d538eaecf","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3bb2937ccccc737d538eaecf: start","job_status":"start","queue_duration_s":2.20635,"scheduling_latency_s":2.20638}1895vm-test-run-nixbot-gitlab> gitlab # [  211.115796] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:10 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/fd1fc7ecbe895687b4904ab556cca5a4e96ea8fb HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 4061896vm-test-run-nixbot-gitlab> gitlab # [  211.119222] gitaly[1392]: time="2026-07-04T12:39:09.682Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7M1B9ZGTCBRMH7GA6F43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-07-04T12:39:19.033" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:09.674" grpc.time_ms=7.485 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1897vm-test-run-nixbot-gitlab> gitlab # [  211.141373] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.300Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3bb2937ccccc737d538eaecf","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3bb2937ccccc737d538eaecf: done: 0.017922 sec","job_status":"done","queue_duration_s":2.20635,"scheduling_latency_s":2.20638,"redis_calls":13,"redis_duration_s":0.003033,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000968,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002065,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012034,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017922,"completed_at":"2026-07-04T12:39:09.300Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1898vm-test-run-nixbot-gitlab> gitlab # [  211.213175] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.301Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5a0bc23f7a6a2adacbb92ca6","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5a0bc23f7a6a2adacbb92ca6: start","job_status":"start","queue_duration_s":2.225778,"scheduling_latency_s":2.225802}1899vm-test-run-nixbot-gitlab> gitlab # [  211.221312] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.318Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5a0bc23f7a6a2adacbb92ca6","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5a0bc23f7a6a2adacbb92ca6: done: 0.016298 sec","job_status":"done","queue_duration_s":2.225778,"scheduling_latency_s":2.225802,"redis_calls":13,"redis_duration_s":0.0029419999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000911,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002031,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010819,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016298,"completed_at":"2026-07-04T12:39:09.318Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1900vm-test-run-nixbot-gitlab> gitlab # [  211.320226] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.320Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e8d5082541230ce2170c3b03","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e8d5082541230ce2170c3b03: start","job_status":"start","queue_duration_s":2.243974,"scheduling_latency_s":2.243995}1901vm-test-run-nixbot-gitlab> gitlab # [  211.321754] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.339Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e8d5082541230ce2170c3b03","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e8d5082541230ce2170c3b03: done: 0.018946 sec","job_status":"done","queue_duration_s":2.243974,"scheduling_latency_s":2.243995,"redis_calls":13,"redis_duration_s":0.00407,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001622,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002448,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011453,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018946,"completed_at":"2026-07-04T12:39:09.339Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1902vm-test-run-nixbot-gitlab> gitlab # [  211.392164] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.340Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f6e1202cef60b193ec2b9181","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f6e1202cef60b193ec2b9181: start","job_status":"start","queue_duration_s":2.264534,"scheduling_latency_s":2.264558}1903vm-test-run-nixbot-gitlab> gitlab # [  211.393429] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.360Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f6e1202cef60b193ec2b9181","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f6e1202cef60b193ec2b9181: done: 0.019536 sec","job_status":"done","queue_duration_s":2.264534,"scheduling_latency_s":2.264558,"redis_calls":13,"redis_duration_s":0.0032530000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001059,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002194,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011396,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019536,"completed_at":"2026-07-04T12:39:09.360Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1904vm-test-run-nixbot-gitlab> gitlab # [  211.506948] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.362Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ac82e7f5c34875e86071b10","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ac82e7f5c34875e86071b10: start","job_status":"start","queue_duration_s":2.285975,"scheduling_latency_s":2.285998}1905vm-test-run-nixbot-gitlab> gitlab # [  211.508215] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.378Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ac82e7f5c34875e86071b10","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ac82e7f5c34875e86071b10: done: 0.016476 sec","job_status":"done","queue_duration_s":2.285975,"scheduling_latency_s":2.285998,"redis_calls":13,"redis_duration_s":0.003365,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001178,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002187,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010259,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016476,"completed_at":"2026-07-04T12:39:09.378Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1906vm-test-run-nixbot-gitlab> gitlab # [  211.527075] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.379Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"19cf23f4932c66e6b7d681a6","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-19cf23f4932c66e6b7d681a6: start","job_status":"start","queue_duration_s":2.303752,"scheduling_latency_s":2.303774}1907vm-test-run-nixbot-gitlab> gitlab # [  211.547107] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.395Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"19cf23f4932c66e6b7d681a6","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-19cf23f4932c66e6b7d681a6: done: 0.015822 sec","job_status":"done","queue_duration_s":2.303752,"scheduling_latency_s":2.303774,"redis_calls":13,"redis_duration_s":0.003015,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000904,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002111,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01003,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015822,"completed_at":"2026-07-04T12:39:09.395Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1908vm-test-run-nixbot-gitlab> gitlab # [  211.626343] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.397Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1b5ed5c6072515dc0cbdfa74","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1b5ed5c6072515dc0cbdfa74: start","job_status":"start","queue_duration_s":2.321037,"scheduling_latency_s":2.321057}1909vm-test-run-nixbot-gitlab> gitlab # [  211.627844] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.413Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1b5ed5c6072515dc0cbdfa74","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1b5ed5c6072515dc0cbdfa74: done: 0.016369 sec","job_status":"done","queue_duration_s":2.321037,"scheduling_latency_s":2.321057,"redis_calls":13,"redis_duration_s":0.002971,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000886,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002085,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009926,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016369,"completed_at":"2026-07-04T12:39:09.413Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1910vm-test-run-nixbot-gitlab> gitlab # [  211.698190] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.414Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"204149deed0ec9baefe41d8a","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-204149deed0ec9baefe41d8a: start","job_status":"start","queue_duration_s":2.338718,"scheduling_latency_s":2.338739}1911vm-test-run-nixbot-gitlab> gitlab # [  211.708083] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.430Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"204149deed0ec9baefe41d8a","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-204149deed0ec9baefe41d8a: done: 0.015431 sec","job_status":"done","queue_duration_s":2.338718,"scheduling_latency_s":2.338739,"redis_calls":13,"redis_duration_s":0.003045,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000888,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002157,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009958,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015431,"completed_at":"2026-07-04T12:39:09.430Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1912vm-test-run-nixbot-gitlab> gitlab # [  211.819113] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.431Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0681044945a61f899d832b10","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0681044945a61f899d832b10: start","job_status":"start","queue_duration_s":2.35547,"scheduling_latency_s":2.355492}1913vm-test-run-nixbot-gitlab> gitlab # [  211.820345] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.450Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0681044945a61f899d832b10","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0681044945a61f899d832b10: done: 0.018762 sec","job_status":"done","queue_duration_s":2.35547,"scheduling_latency_s":2.355492,"redis_calls":13,"redis_duration_s":0.003164,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001127,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002037,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012701,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018762,"completed_at":"2026-07-04T12:39:09.450Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1914vm-test-run-nixbot-gitlab> gitlab # [  211.851796] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.452Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f077ad1e7f4fcf1fbaa22783","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f077ad1e7f4fcf1fbaa22783: start","job_status":"start","queue_duration_s":2.376324,"scheduling_latency_s":2.376352}1915vm-test-run-nixbot-gitlab> gitlab # [  211.871826] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.468Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f077ad1e7f4fcf1fbaa22783","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f077ad1e7f4fcf1fbaa22783: done: 0.016313 sec","job_status":"done","queue_duration_s":2.376324,"scheduling_latency_s":2.376352,"redis_calls":13,"redis_duration_s":0.003312,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000944,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002368,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01031,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016313,"completed_at":"2026-07-04T12:39:09.468Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1916vm-test-run-nixbot-gitlab> gitlab # [  211.966335] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.470Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"216b329e1ae8e42924439d3d","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-216b329e1ae8e42924439d3d: start","job_status":"start","queue_duration_s":2.393962,"scheduling_latency_s":2.393983}1917vm-test-run-nixbot-gitlab> gitlab # [  211.967847] gitaly[1392]: time="2026-07-04T12:39:09.864Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7M7G3S61K9D60W7H3XDF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T12:39:09.852" grpc.time_ms=11.802 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc1918vm-test-run-nixbot-gitlab> gitlab # [  211.973390] gitaly[1392]: time="2026-07-04T12:39:09.940Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7M9CWYEC0K36A2N1XQH5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T12:39:09.928" pid=1392 remote_ip="2001:db8:1::1" request_sha=083c43590fffa527704b32b8836424e8ed055017 response_bytes=149 span.kind=server system=grpc1919vm-test-run-nixbot-gitlab> gitlab # [  211.990080] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.488Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"216b329e1ae8e42924439d3d","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-216b329e1ae8e42924439d3d: done: 0.017976 sec","job_status":"done","queue_duration_s":2.393962,"scheduling_latency_s":2.393983,"redis_calls":13,"redis_duration_s":0.003222,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001078,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002144,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011182,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017976,"completed_at":"2026-07-04T12:39:09.488Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1920vm-test-run-nixbot-gitlab> gitlab # [  212.103930] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.489Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7528cbba2165d4c5468b496e","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7528cbba2165d4c5468b496e: start","job_status":"start","queue_duration_s":2.413723,"scheduling_latency_s":2.413753}1921vm-test-run-nixbot-gitlab> gitlab # [  212.105172] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.510Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7528cbba2165d4c5468b496e","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7528cbba2165d4c5468b496e: done: 0.020803 sec","job_status":"done","queue_duration_s":2.413723,"scheduling_latency_s":2.413753,"redis_calls":13,"redis_duration_s":0.003672,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001237,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002435,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01257,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020803,"completed_at":"2026-07-04T12:39:09.510Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1922vm-test-run-nixbot-gitlab> gitlab # [  212.180473] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.512Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd6eb258d8d460c69b21087d","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd6eb258d8d460c69b21087d: start","job_status":"start","queue_duration_s":2.436025,"scheduling_latency_s":2.436053}1923vm-test-run-nixbot-gitlab> gitlab # [  212.182101] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.529Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd6eb258d8d460c69b21087d","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd6eb258d8d460c69b21087d: done: 0.017397 sec","job_status":"done","queue_duration_s":2.436025,"scheduling_latency_s":2.436053,"redis_calls":13,"redis_duration_s":0.0036190000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000968,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002651,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010132,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017397,"completed_at":"2026-07-04T12:39:09.529Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1924vm-test-run-nixbot-gitlab> gitlab # [  212.255186] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.530Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"07bf139ef949f1c9807e9fa8","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-07bf139ef949f1c9807e9fa8: start","job_status":"start","queue_duration_s":2.454721,"scheduling_latency_s":2.454743}1925vm-test-run-nixbot-gitlab> gitlab # [  212.256425] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.550Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"07bf139ef949f1c9807e9fa8","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-07bf139ef949f1c9807e9fa8: done: 0.019302 sec","job_status":"done","queue_duration_s":2.454721,"scheduling_latency_s":2.454743,"redis_calls":13,"redis_duration_s":0.0038900000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000971,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002919,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012355,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019302,"completed_at":"2026-07-04T12:39:09.550Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1926vm-test-run-nixbot-gitlab> gitlab # [  212.341894] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.551Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4f4db5070a681457347b54bd","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4f4db5070a681457347b54bd: start","job_status":"start","queue_duration_s":2.475514,"scheduling_latency_s":2.475541}1927vm-test-run-nixbot-gitlab> gitlab # [  212.343255] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.568Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4f4db5070a681457347b54bd","created_at":"2026-07-04T12:39:07.009Z","trace_propagation_headers":{"sentry-trace":"084a7704b4de467b8859bb6e67fb44e4-67703bbecbce4d74","baggage":"sentry-trace_id=084a7704b4de467b8859bb6e67fb44e4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73dd27555fde88f8e545fe303f3230","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:07.076Z","job_size_bytes":2,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4f4db5070a681457347b54bd: done: 0.016682 sec","job_status":"done","queue_duration_s":2.475514,"scheduling_latency_s":2.475541,"redis_calls":13,"redis_duration_s":0.002952,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000903,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002049,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010341,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016682,"completed_at":"2026-07-04T12:39:09.568Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1928vm-test-run-nixbot-gitlab> gitlab # [  212.379228] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.570Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"54885c185fe5d1a10e5a9458","created_at":"2026-07-04T12:39:08.472Z","trace_propagation_headers":{"sentry-trace":"d311a4d5f6d54ca6a50d968410119c29-ece7934b7ced4cc1","baggage":"sentry-trace_id=d311a4d5f6d54ca6a50d968410119c29,sentry-release=Unknown"},"correlation_id":"01KWPJ7E1BWTSTSTFPBQNWTB5G","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:08.476Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-54885c185fe5d1a10e5a9458: start","job_status":"start","queue_duration_s":1.09358,"scheduling_latency_s":1.093601}1929vm-test-run-nixbot-gitlab> gitlab # [  212.448417] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:09.657Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"54885c185fe5d1a10e5a9458","created_at":"2026-07-04T12:39:08.472Z","trace_propagation_headers":{"sentry-trace":"d311a4d5f6d54ca6a50d968410119c29-ece7934b7ced4cc1","baggage":"sentry-trace_id=d311a4d5f6d54ca6a50d968410119c29,sentry-release=Unknown"},"correlation_id":"01KWPJ7E1BWTSTSTFPBQNWTB5G","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:08.476Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-54885c185fe5d1a10e5a9458: done: 0.087299 sec","job_status":"done","queue_duration_s":1.09358,"scheduling_latency_s":1.093601,"redis_calls":11,"redis_duration_s":0.003382,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000613,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002173,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000596,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.009,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.058599,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000508444,"duration_s":0.087299,"completed_at":"2026-07-04T12:39:09.657Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.023663,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1930vm-test-run-nixbot-gitlab> gitlab # [  212.536913] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:10.517Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"7aec0dc56a829379112fdfb1","created_at":"2026-07-04T12:39:10.465Z","trace_propagation_headers":{"sentry-trace":"f226ec03a3f9415aaf4db2bbcd837462-dbebdc4935004b8d","baggage":"sentry-trace_id=f226ec03a3f9415aaf4db2bbcd837462,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPJ7MKWM7MDBZE20YRZ67BK","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:10.472Z","job_size_bytes":65,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-7aec0dc56a829379112fdfb1: start","job_status":"start","queue_duration_s":0.045081,"scheduling_latency_s":0.045124}1931vm-test-run-nixbot-gitlab> gitlab # [  212.538165] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:10.568Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"7aec0dc56a829379112fdfb1","created_at":"2026-07-04T12:39:10.465Z","trace_propagation_headers":{"sentry-trace":"f226ec03a3f9415aaf4db2bbcd837462-dbebdc4935004b8d","baggage":"sentry-trace_id=f226ec03a3f9415aaf4db2bbcd837462,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPJ7MKWM7MDBZE20YRZ67BK","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:10.472Z","job_size_bytes":65,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-7aec0dc56a829379112fdfb1: done: 0.051105 sec","job_status":"done","queue_duration_s":0.045081,"scheduling_latency_s":0.045124,"redis_calls":19,"redis_duration_s":0.005197,"redis_read_bytes":145,"redis_write_bytes":2300,"redis_feature_flag_calls":11,"redis_feature_flag_duration_s":0.00312,"redis_feature_flag_read_bytes":138,"redis_feature_flag_write_bytes":1482,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001538,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000539,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02952,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000368203,"duration_s":0.051105,"completed_at":"2026-07-04T12:39:10.568Z","load_balancing_strategy":"primary","db_duration_s":0.011567,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1932vm-test-run-nixbot-gitlab> gitlab # [  212.607565] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:10.569Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"eecee6438361a91767d2a894","created_at":"2026-07-04T12:39:10.473Z","trace_propagation_headers":{"sentry-trace":"f226ec03a3f9415aaf4db2bbcd837462-dbebdc4935004b8d","baggage":"sentry-trace_id=f226ec03a3f9415aaf4db2bbcd837462,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPJ7MKWM7MDBZE20YRZ67BK","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:10.479Z","job_size_bytes":65,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-eecee6438361a91767d2a894: start","job_status":"start","queue_duration_s":0.089877,"scheduling_latency_s":0.089897}1933vm-test-run-nixbot-gitlab> gitlab # [  212.609148] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:10.632Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"eecee6438361a91767d2a894","created_at":"2026-07-04T12:39:10.473Z","trace_propagation_headers":{"sentry-trace":"f226ec03a3f9415aaf4db2bbcd837462-dbebdc4935004b8d","baggage":"sentry-trace_id=f226ec03a3f9415aaf4db2bbcd837462,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPJ7MKWM7MDBZE20YRZ67BK","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:10.479Z","job_size_bytes":65,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-eecee6438361a91767d2a894: done: 0.063037 sec","job_status":"done","queue_duration_s":0.089877,"scheduling_latency_s":0.089897,"redis_calls":15,"redis_duration_s":0.004524,"redis_read_bytes":142,"redis_write_bytes":1431,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001007,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":431,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002086,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001431,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038099,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000378539,"duration_s":0.063037,"completed_at":"2026-07-04T12:39:10.632Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011446,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1934vm-test-run-nixbot-gitlab> gitlab # [  212.683085] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:10.634Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"9622985a8b54a60c689c6316","created_at":"2026-07-04T12:39:10.589Z","trace_propagation_headers":{"sentry-trace":"f226ec03a3f9415aaf4db2bbcd837462-dbebdc4935004b8d","baggage":"sentry-trace_id=f226ec03a3f9415aaf4db2bbcd837462,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPJ7MKWM7MDBZE20YRZ67BK","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:10.595Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-9622985a8b54a60c689c6316: start","job_status":"start","queue_duration_s":0.038605,"scheduling_latency_s":0.038628}1935vm-test-run-nixbot-gitlab> gitlab # [  212.694102] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:10.773Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"9622985a8b54a60c689c6316","created_at":"2026-07-04T12:39:10.589Z","trace_propagation_headers":{"sentry-trace":"f226ec03a3f9415aaf4db2bbcd837462-dbebdc4935004b8d","baggage":"sentry-trace_id=f226ec03a3f9415aaf4db2bbcd837462,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPJ7MKWM7MDBZE20YRZ67BK","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:10.595Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-9622985a8b54a60c689c6316: done: 0.139705 sec","job_status":"done","queue_duration_s":0.038605,"scheduling_latency_s":0.038628,"gitaly_calls":3,"gitaly_duration_s":0.007886,"redis_calls":53,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.010294000000000001,"redis_read_bytes":235,"redis_write_bytes":7158,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001172,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.002112,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_feature_flag_calls":7,"redis_feature_flag_duration_s":0.001557,"redis_feature_flag_read_bytes":134,"redis_feature_flag_write_bytes":992,"redis_queues_calls":2,"redis_queues_duration_s":0.000291,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1132,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.001861,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":840,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000588,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000457,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001084,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":34,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":34,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":22,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.006,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.096243,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.113546554,"duration_s":0.139705,"completed_at":"2026-07-04T12:39:10.773Z","load_balancing_strategy":"primary","db_duration_s":0.02952,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1936vm-test-run-nixbot-gitlab> gitlab # [  212.778178] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:10.775Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"e17665c244644ab28b4f4d70","created_at":"2026-07-04T12:39:10.696Z","trace_propagation_headers":{"sentry-trace":"f226ec03a3f9415aaf4db2bbcd837462-4ef01d9a61e040c2","baggage":"sentry-trace_id=f226ec03a3f9415aaf4db2bbcd837462,sentry-release=Unknown"},"correlation_id":"01KWPJ7MKWM7MDBZE20YRZ67BK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:10.700Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-e17665c244644ab28b4f4d70: start","job_status":"start","queue_duration_s":0.075615,"scheduling_latency_s":0.075641}1937vm-test-run-nixbot-gitlab> gitlab # [  212.803263] gitaly[1392]: time="2026-07-04T12:39:09.940Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7M9CWYEC0K36A2N1XQH5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T12:39:09.928" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc1938vm-test-run-nixbot-gitlab> gitlab # [  212.871575] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:10.805Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"e17665c244644ab28b4f4d70","created_at":"2026-07-04T12:39:10.696Z","trace_propagation_headers":{"sentry-trace":"f226ec03a3f9415aaf4db2bbcd837462-4ef01d9a61e040c2","baggage":"sentry-trace_id=f226ec03a3f9415aaf4db2bbcd837462,sentry-release=Unknown"},"correlation_id":"01KWPJ7MKWM7MDBZE20YRZ67BK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:10.700Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-e17665c244644ab28b4f4d70: done: 0.029407 sec","job_status":"done","queue_duration_s":0.075615,"scheduling_latency_s":0.075641,"redis_calls":10,"redis_duration_s":0.002316,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000424,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00144,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000452,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019289,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000353396,"duration_s":0.029407,"completed_at":"2026-07-04T12:39:10.805Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003292,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1939vm-test-run-nixbot-gitlab> gitlab # [  212.893157] gitaly[1392]: time="2026-07-04T12:39:09.944Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7M9CWYEC0K36A2N1XQH5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T12:39:09.928" grpc.time_ms=12.946 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc1940vm-test-run-nixbot-gitlab> gitlab # [  212.967089] gitaly[1392]: time="2026-07-04T12:39:10.008Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7MBS231BP8F9K6KVGRZ4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T12:39:09.992" pid=1392 remote_ip="2001:db8:1::1" request_sha=29b706d3d3bcafe102d92c9946d9b9674ec042f8 response_bytes=373 span.kind=server system=grpc1941vm-test-run-nixbot-gitlab> gitlab # [  212.968369] gitaly[1392]: time="2026-07-04T12:39:10.008Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1235 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7MBS231BP8F9K6KVGRZ4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T12:39:09.992" negotiation.command=fetch negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc1942vm-test-run-nixbot-gitlab> gitlab # [  212.969158] gitaly[1392]: time="2026-07-04T12:39:10.009Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1235 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7MBS231BP8F9K6KVGRZ4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=9 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T12:39:09.992" grpc.time_ms=16.387 negotiation.command=fetch negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc1943vm-test-run-nixbot-gitlab> gitlab # [  213.036536] gitaly[1392]: time="2026-07-04T12:39:10.108Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=4 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7MF1DG6J1KY1K63MTX9G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T12:39:10.092" grpc.time_ms=16.145 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc1944vm-test-run-nixbot-gitlab> gitlab # [  213.038089] gitaly[1392]: time="2026-07-04T12:39:10.171Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7MH07GVNHRWY8E0M3RSQ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T12:39:10.162" pid=1392 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc1945vm-test-run-nixbot-gitlab> gitlab # [  213.038954] gitaly[1392]: time="2026-07-04T12:39:10.171Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7MH07GVNHRWY8E0M3RSQ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T12:39:10.162" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc1946vm-test-run-nixbot-gitlab> gitlab # [  213.040119] gitaly[1392]: time="2026-07-04T12:39:10.172Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7MH07GVNHRWY8E0M3RSQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T12:39:10.162" grpc.time_ms=9.303 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc1947vm-test-run-nixbot-gitlab> gitlab # [  213.108222] gitaly[1392]: time="2026-07-04T12:39:10.250Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7MKWM7MDBZE20YRZ67BK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T12:39:40.048" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:10.249" grpc.time_ms=0.661 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1948vm-test-run-nixbot-gitlab> gitlab # [  213.109427] gitaly[1392]: time="2026-07-04T12:39:10.292Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=647 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7MKWM7MDBZE20YRZ67BK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-07-04T12:39:40.083" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T12:39:10.283" grpc.time_ms=8.425 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1949vm-test-run-nixbot-gitlab> gitlab # [  213.110280] gitaly[1392]: time="2026-07-04T12:39:10.305Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7MKWM7MDBZE20YRZ67BK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-04T12:39:20.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T12:39:10.299" grpc.time_ms=5.916 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1950vm-test-run-nixbot-gitlab> gitlab # [  213.117241] gitaly[1392]: time="2026-07-04T12:39:10.352Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7MKWM7MDBZE20YRZ67BK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-04T12:39:20.061" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T12:39:10.351" grpc.time_ms=0.258 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1951vm-test-run-nixbot-gitlab> gitlab # [  213.134400] gitaly[1392]: time="2026-07-04T12:39:10.365Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7MKWM7MDBZE20YRZ67BK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-04T12:39:20.068" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T12:39:10.358" grpc.time_ms=6.531 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1952vm-test-run-nixbot-gitlab> gitlab # [  213.156066] gitaly[1392]: time="2026-07-04T12:39:10.407Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7MKWM7MDBZE20YRZ67BK grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4159 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T12:39:10.406" grpc.time_ms=0.348 pid=1392 span.kind=server system=grpc1953vm-test-run-nixbot-gitlab> gitlab # [  213.171498] gitaly[1392]: time="2026-07-04T12:39:10.433Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7MKWM7MDBZE20YRZ67BK grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4161 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T12:39:10.432" grpc.time_ms=0.417 pid=1392 span.kind=server system=grpc1954vm-test-run-nixbot-gitlab> gitlab # [  213.253629] gitaly[1392]: time="2026-07-04T12:39:10.454Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7MKWM7MDBZE20YRZ67BK diskcache=eddb1b11-e2cf-4e32-8d2e-770de43e9609 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-04T12:39:20.077" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T12:39:10.367" pid=1392 ref=refs/keep-around/fd1fc7ecbe895687b4904ab556cca5a4e96ea8fb remote_ip="2001:db8:1::1" revision=fd1fc7ecbe895687b4904ab556cca5a4e96ea8fb span.kind=server system=grpc username=root1955vm-test-run-nixbot-gitlab> gitlab # [  213.255307] gitaly[1392]: time="2026-07-04T12:39:10.455Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=66 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5139 command.oublock=8 command.real_time_ms=69 command.spawn_token_fork_ms=0 command.system_time_ms=43 command.user_time_ms=23 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7MKWM7MDBZE20YRZ67BK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-04T12:39:20.077" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T12:39:10.367" grpc.time_ms=87.513 pid=1392 ref=refs/keep-around/fd1fc7ecbe895687b4904ab556cca5a4e96ea8fb remote_ip="2001:db8:1::1" revision=fd1fc7ecbe895687b4904ab556cca5a4e96ea8fb span.kind=server system=grpc username=root1956vm-test-run-nixbot-gitlab> gitlab # [  213.256113] gitaly[1392]: time="2026-07-04T12:39:10.459Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7MKWM7MDBZE20YRZ67BK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T12:39:40.258" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:10.458" grpc.time_ms=0.391 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1957vm-test-run-nixbot-gitlab> gitlab # [  213.256956] gitaly[1392]: time="2026-07-04T12:39:10.711Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7MKWM7MDBZE20YRZ67BK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-04T12:39:40.710" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:10.710" grpc.time_ms=0.875 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1958vm-test-run-nixbot-gitlab> gitlab # [  213.269479] gitaly[1392]: time="2026-07-04T12:39:10.741Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7MKWM7MDBZE20YRZ67BK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T12:39:40.740" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:10.740" grpc.time_ms=0.607 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1959vm-test-run-nixbot-gitlab> gitlab # [  213.340739] gitaly[1392]: time="2026-07-04T12:39:10.763Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7MKWM7MDBZE20YRZ67BK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T12:39:40.762" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:10.762" grpc.time_ms=0.848 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1960vm-test-run-nixbot-gitlab> gitlab # [  213.640168] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 6805666271 wd_nsec: 68056629471961vm-test-run-nixbot-gitlab> gitlab # [  213.380559] systemd[1]: Startup finished in 1.447s (kernel) + 4.425s (initrd) + 3min 27.507s (userspace) = 3min 33.379s.1962vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for unit nixbot.service, in 9.27 seconds)1963vm-test-run-nixbot-gitlab> gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1964vm-test-run-nixbot-gitlab> gitlab # [  213.489931] nixbot[3144]: INFO:     ::ffff:127.0.0.1:59424 - "GET /health HTTP/1.1" 200 OK1965vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.04 seconds)1966vm-test-run-nixbot-gitlab> subtest: project discovered and webhook registered1967vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1968vm-test-run-nixbot-gitlab> gitlab # [  213.501236] systemd[1]: Started Nix Daemon.1969vm-test-run-nixbot-gitlab> gitlab # [  213.557577] nix-daemon[3304]: accepted connection from pid 3296, user nixbot1970vm-test-run-nixbot-gitlab> gitlab # [  213.561410] nix-daemon[3304]: accepted connection from pid 3295, user nixbot1971vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.08 seconds)1972vm-test-run-nixbot-gitlab> [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-07-04T12:39:09.274Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': False, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]1973vm-test-run-nixbot-gitlab> (finished: subtest: project discovered and webhook registered, in 0.08 seconds)1974vm-test-run-nixbot-gitlab> subtest: push delivers the webhook1975vm-test-run-nixbot-gitlab> gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1976vm-test-run-nixbot-gitlab> gitlab # [  213.571203] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:14 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "curl/8.20.0" 471977vm-test-run-nixbot-gitlab> gitlab # [  213.655293] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:14 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 261978vm-test-run-nixbot-gitlab> gitlab # [  213.761716] gitaly[1392]: time="2026-07-04T12:39:14.713Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7RXGYVS5PV4CM50R1QS4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T12:39:14.698" grpc.time_ms=13.876 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1979vm-test-run-nixbot-gitlab> gitlab # [  213.765269] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:14 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 201 "" "git/2.54.0" 1041980vm-test-run-nixbot-gitlab> gitlab # Enumerating objects: 5, done.1981vm-test-run-nixbot-gitlab> gitlab # Counting objects:  20% (1/5)Counting objects:  40% (2/5)Counting objects:  60% (3/5)Counting objects:  80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1982vm-test-run-nixbot-gitlab> gitlab # Delta compression using up to 4 threads1983vm-test-run-nixbot-gitlab> gitlab # Compressing objects:  50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1984vm-test-run-nixbot-gitlab> gitlab # Writing objects:  33% (1/3)Writing objects:  66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 273 bytes | 54.00 KiB/s, done.1985vm-test-run-nixbot-gitlab> gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1986vm-test-run-nixbot-gitlab> gitlab # [  214.074042] gitaly[1392]: time="2026-07-04T12:39:15.025Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7S3JSAJFY5JKVKS23Q17 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-07-04T12:39:25.004" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=253 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:15.004" grpc.time_ms=20.113 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1987vm-test-run-nixbot-gitlab> gitlab # [  214.150264] gitaly[1392]: time="2026-07-04T12:39:15.101Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=330 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7S3JSAJFY5JKVKS23Q17 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-07-04T12:39:45.093" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-07-04T12:39:15.094" grpc.time_ms=7.198 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1988vm-test-run-nixbot-gitlab> gitlab # [  214.174276] gitaly[1392]: time="2026-07-04T12:39:15.125Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KWPJ7S3JSAJFY5JKVKS23Q17 duration_ms=178ns method=POST pid=1392 status=200 url="http://unix/api/v4/internal/allowed"1989vm-test-run-nixbot-gitlab> gitlab # [  214.212358] gitaly[1392]: time="2026-07-04T12:39:15.163Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KWPJ7S3JSAJFY5JKVKS23Q17 duration_ms=37ns method=POST pid=1392 status=200 url="http://unix/api/v4/internal/pre_receive"1990vm-test-run-nixbot-gitlab> gitlab # [  214.217374] gitaly[1392]: time="2026-07-04T12:39:15.168Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7S3JSAJFY5JKVKS23Q17 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5322 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T12:39:14.946" grpc.time_ms=217.551 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1991vm-test-run-nixbot-gitlab> gitlab # [  214.266702] gitaly[1392]: time="2026-07-04T12:39:15.218Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7S3JSAJFY5JKVKS23Q17 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4792 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T12:39:15.216" grpc.time_ms=1.928 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1992vm-test-run-nixbot-gitlab> gitlab # [  214.316429] gitaly[1392]: time="2026-07-04T12:39:15.268Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7S3JSAJFY5JKVKS23Q17 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4791 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T12:39:15.267" grpc.time_ms=0.515 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1993vm-test-run-nixbot-gitlab> gitlab # [  214.363441] gitaly[1392]: time="2026-07-04T12:39:15.315Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7S3JSAJFY5JKVKS23Q17 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4793 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T12:39:15.313" grpc.time_ms=1.116 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1994vm-test-run-nixbot-gitlab> gitlab # [  214.435348] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:15.386Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","fd1fc7ecbe895687b4904ab556cca5a4e96ea8fb 98211a6541adb8d86116a7680ad181c0d8549f29 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"d4a749e13f0dcc8b016972d3","created_at":"2026-07-04T12:39:15.381Z","trace_propagation_headers":{"sentry-trace":"efaf3741cf434f84ae376dc2f2e06b7a-6f84f23dcbe0470d","baggage":"sentry-trace_id=efaf3741cf434f84ae376dc2f2e06b7a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPJ7S3JSAJFY5JKVKS23Q17","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:15.383Z","job_size_bytes":185,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-d4a749e13f0dcc8b016972d3: start","job_status":"start","queue_duration_s":0.002688,"scheduling_latency_s":0.00273}1995vm-test-run-nixbot-gitlab> gitlab # [  214.453223] gitaly[1392]: time="2026-07-04T12:39:15.404Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KWPJ7S3JSAJFY5JKVKS23Q17 duration_ms=46ns method=POST pid=1392 status=200 url="http://unix/api/v4/internal/post_receive"1996vm-test-run-nixbot-gitlab> gitlab # [  214.456606] gitaly[1392]: time="2026-07-04T12:39:15.406Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7S3JSAJFY5JKVKS23Q17 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4816 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T12:39:15.357" grpc.time_ms=49.007 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1997vm-test-run-nixbot-gitlab> gitlab # [  214.467454] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:15.418Z","message":"changes: \"fd1fc7ecbe895687b4904ab556cca5a4e96ea8fb 98211a6541adb8d86116a7680ad181c0d8549f29 refs/heads/master\\n\""}1998vm-test-run-nixbot-gitlab> gitlab # [  214.491218] gitaly[1392]: time="2026-07-04T12:39:15.442Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7S3JSAJFY5JKVKS23Q17 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-04T12:39:25.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T12:39:15.434" grpc.time_ms=7.186 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1999vm-test-run-nixbot-gitlab> gitlab # [  214.497953] gitaly[1392]: time="2026-07-04T12:39:15.449Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7S3JSAJFY5JKVKS23Q17 diskcache=beb78ed1-f9f9-43df-8e3b-1a4b12781395 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T12:39:14.864" pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root2000vm-test-run-nixbot-gitlab> gitlab # [  214.499429] gitaly[1392]: time="2026-07-04T12:39:15.451Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=223 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13682 command.oublock=32 command.real_time_ms=576 command.spawn_token_fork_ms=0 command.system_time_ms=167 command.user_time_ms=56 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7S3JSAJFY5JKVKS23Q17 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=605 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T12:39:14.864" grpc.time_ms=586.481 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root2001vm-test-run-nixbot-gitlab> gitlab # [  214.500857] gitlab-workhorse[1293]: time="2026-07-04T12:39:15Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=522002vm-test-run-nixbot-gitlab> gitlab # [  214.501899] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:15 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 6492003vm-test-run-nixbot-gitlab> gitlab # To http://gitlab/root/test-flake.git2004vm-test-run-nixbot-gitlab> gitlab #    fd1fc7e..98211a6  master -> master2005vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.00 seconds)2006vm-test-run-nixbot-gitlab> gitlab: must succeed: git -C /tmp/test-flake rev-parse master2007vm-test-run-nixbot-gitlab> gitlab # [  214.569096] gitaly[1392]: time="2026-07-04T12:39:15.520Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7S3JSAJFY5JKVKS23Q17 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-04T12:39:25.083" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T12:39:15.513" grpc.time_ms=6.873 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2008vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.03 seconds)2009vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=98211a6541adb8d86116a7680ad181c0d8549f29'2010vm-test-run-nixbot-gitlab> gitlab # [  214.592724] gitaly[1392]: time="2026-07-04T12:39:15.544Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7S3JSAJFY5JKVKS23Q17 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-07-04T12:39:25.107" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T12:39:15.537" grpc.time_ms=6.264 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2011vm-test-run-nixbot-gitlab> gitlab # [  214.604330] gitaly[1392]: time="2026-07-04T12:39:15.555Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=656 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7S3JSAJFY5JKVKS23Q17 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-07-04T12:39:45.047" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:15.548" grpc.time_ms=7.283 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2012vm-test-run-nixbot-gitlab> gitlab # [  214.614658] nixbot[3144]: INFO:     ::ffff:127.0.0.1:59440 - "GET /api/repos/gitlab/root/test-flake/builds?commit=98211a6541adb8d86116a7680ad181c0d8549f29 HTTP/1.1" 200 OK2013vm-test-run-nixbot-gitlab> gitlab # [  214.620105] gitaly[1392]: time="2026-07-04T12:39:15.571Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=345 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7S3JSAJFY5JKVKS23Q17 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-07-04T12:39:25.137" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-07-04T12:39:15.567" grpc.time_ms=4.001 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2014vm-test-run-nixbot-gitlab> gitlab # [  214.632267] gitaly[1392]: time="2026-07-04T12:39:15.583Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7S3JSAJFY5JKVKS23Q17 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-04T12:39:25.148" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T12:39:15.578" grpc.time_ms=5.23 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2015vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=98211a6541adb8d86116a7680ad181c0d8549f29', in 0.10 seconds)2016vm-test-run-nixbot-gitlab> []2017vm-test-run-nixbot-gitlab> gitlab # [  214.990399] gitaly[1392]: time="2026-07-04T12:39:15.941Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7S3JSAJFY5JKVKS23Q17 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-07-04T12:39:25.099" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:15.939" grpc.time_ms=1.616 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2018vm-test-run-nixbot-gitlab> gitlab # [  215.011848] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:15.963Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","fd1fc7ecbe895687b4904ab556cca5a4e96ea8fb 98211a6541adb8d86116a7680ad181c0d8549f29 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"d4a749e13f0dcc8b016972d3","created_at":"2026-07-04T12:39:15.381Z","trace_propagation_headers":{"sentry-trace":"efaf3741cf434f84ae376dc2f2e06b7a-6f84f23dcbe0470d","baggage":"sentry-trace_id=efaf3741cf434f84ae376dc2f2e06b7a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPJ7S3JSAJFY5JKVKS23Q17","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-04T12:39:15.383Z","job_size_bytes":185,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-d4a749e13f0dcc8b016972d3: done: 0.577 sec","job_status":"done","queue_duration_s":0.002688,"scheduling_latency_s":0.00273,"gitaly_calls":7,"gitaly_duration_s":0.123202,"redis_calls":112,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.21845399999999998,"redis_read_bytes":350,"redis_write_bytes":13069,"redis_cache_calls":9,"redis_cache_duration_s":0.002197,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.1921,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":323,"redis_queues_calls":8,"redis_queues_duration_s":0.001537,"redis_queues_read_bytes":8,"redis_queues_write_bytes":6769,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.005787,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1182,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.008055,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.008778,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":29,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":29,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.039,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":29,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.02,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.02,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.039,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.256794,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000625778,"exclusive_lock_hold_duration_s":0.08253563,"duration_s":0.577,"completed_at":"2026-07-04T12:39:15.962Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.045096,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2019vm-test-run-nixbot-gitlab> gitlab # [  215.058630] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:15.965Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"d2051b85be8237a9e9adeb57","created_at":"2026-07-04T12:39:15.522Z","trace_propagation_headers":{"sentry-trace":"efaf3741cf434f84ae376dc2f2e06b7a-326a215a2e9f49d7","baggage":"sentry-trace_id=efaf3741cf434f84ae376dc2f2e06b7a,sentry-release=Unknown"},"correlation_id":"01KWPJ7S3JSAJFY5JKVKS23Q17","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:15.526Z","job_size_bytes":46,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-d2051b85be8237a9e9adeb57: start","job_status":"start","queue_duration_s":0.438767,"scheduling_latency_s":0.438794}2020vm-test-run-nixbot-gitlab> gitlab # [  215.084354] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:15.987Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"d2051b85be8237a9e9adeb57","created_at":"2026-07-04T12:39:15.522Z","trace_propagation_headers":{"sentry-trace":"efaf3741cf434f84ae376dc2f2e06b7a-326a215a2e9f49d7","baggage":"sentry-trace_id=efaf3741cf434f84ae376dc2f2e06b7a,sentry-release=Unknown"},"correlation_id":"01KWPJ7S3JSAJFY5JKVKS23Q17","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:15.526Z","job_size_bytes":46,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-d2051b85be8237a9e9adeb57: done: 0.022037 sec","job_status":"done","queue_duration_s":0.438767,"scheduling_latency_s":0.438794,"redis_calls":9,"redis_duration_s":0.004004,"redis_read_bytes":23,"redis_write_bytes":782,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002215,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000426,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001363,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":304,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011957,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022037,"completed_at":"2026-07-04T12:39:15.987Z","load_balancing_strategy":"primary","db_duration_s":0.002875,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2021vm-test-run-nixbot-gitlab> gitlab # [  215.155329] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:15.989Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"b296b40dc241da1aec9f4fe1","created_at":"2026-07-04T12:39:15.531Z","trace_propagation_headers":{"sentry-trace":"efaf3741cf434f84ae376dc2f2e06b7a-326a215a2e9f49d7","baggage":"sentry-trace_id=efaf3741cf434f84ae376dc2f2e06b7a,sentry-release=Unknown"},"correlation_id":"01KWPJ7S3JSAJFY5JKVKS23Q17","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:15.533Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-b296b40dc241da1aec9f4fe1: start","job_status":"start","queue_duration_s":0.455385,"scheduling_latency_s":0.455406}2022vm-test-run-nixbot-gitlab> gitlab # [  215.227634] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:16.064Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"b296b40dc241da1aec9f4fe1","created_at":"2026-07-04T12:39:15.531Z","trace_propagation_headers":{"sentry-trace":"efaf3741cf434f84ae376dc2f2e06b7a-326a215a2e9f49d7","baggage":"sentry-trace_id=efaf3741cf434f84ae376dc2f2e06b7a,sentry-release=Unknown"},"correlation_id":"01KWPJ7S3JSAJFY5JKVKS23Q17","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:15.533Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-b296b40dc241da1aec9f4fe1: done: 0.075494 sec","job_status":"done","queue_duration_s":0.455385,"scheduling_latency_s":0.455406,"gitaly_calls":1,"gitaly_duration_s":0.042904,"redis_calls":12,"redis_duration_s":0.005077,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002013,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00119,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001874,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020742,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.055959626,"duration_s":0.075494,"completed_at":"2026-07-04T12:39:16.064Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006801,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2023vm-test-run-nixbot-gitlab> gitlab # [  215.250124] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:16.067Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"46304db4ee24c38d9f8dfd4f","created_at":"2026-07-04T12:39:15.627Z","trace_propagation_headers":{"sentry-trace":"efaf3741cf434f84ae376dc2f2e06b7a-326a215a2e9f49d7","baggage":"sentry-trace_id=efaf3741cf434f84ae376dc2f2e06b7a,sentry-release=Unknown"},"correlation_id":"01KWPJ7S3JSAJFY5JKVKS23Q17","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:a821051b6ad56e93a2e6186c7c4663b4d9d3c5fe700cca9ac2c6212086f8e890","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:15.628Z","job_size_bytes":467,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-46304db4ee24c38d9f8dfd4f: start","job_status":"start","queue_duration_s":0.438426,"scheduling_latency_s":0.43845}2024vm-test-run-nixbot-gitlab> gitlab # [  215.329845] gitaly[1392]: time="2026-07-04T12:39:16.048Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7S3JSAJFY5JKVKS23Q17 diskcache=b0cdf4a1-57a3-4319-906b-4a74bd39d1e6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-04T18:39:16.008" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:16.009" pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2025vm-test-run-nixbot-gitlab> gitlab # [  215.331181] gitaly[1392]: time="2026-07-04T12:39:16.049Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=5 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1648 command.oublock=0 command.real_time_ms=44 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7S3JSAJFY5JKVKS23Q17 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-04T18:39:16.008" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:16.009" grpc.time_ms=39.489 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2026vm-test-run-nixbot-gitlab> gitlab # [  215.331719] gitaly[1392]: time="2026-07-04T12:39:16.089Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7S3JSAJFY5JKVKS23Q17 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T12:39:46.088" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:16.089" grpc.time_ms=0.57 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2027vm-test-run-nixbot-gitlab> gitlab # [  215.336893] gitaly[1392]: time="2026-07-04T12:39:16.100Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7S3JSAJFY5JKVKS23Q17 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-04T12:39:26.001" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T12:39:16.091" grpc.time_ms=8.917 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2028vm-test-run-nixbot-gitlab> gitlab # [  215.408953] gitaly[1392]: time="2026-07-04T12:39:16.129Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7S3JSAJFY5JKVKS23Q17 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-04T12:39:46.127" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:16.128" grpc.time_ms=0.964 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2029vm-test-run-nixbot-gitlab> gitlab # [  215.410223] gitaly[1392]: time="2026-07-04T12:39:16.135Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=202 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7S3JSAJFY5JKVKS23Q17 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-07-04T12:39:46.131" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:16.132" grpc.time_ms=3.232 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2030vm-test-run-nixbot-gitlab> gitlab # [  215.411102] gitaly[1392]: time="2026-07-04T12:39:16.182Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7S3JSAJFY5JKVKS23Q17 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-04T12:39:26.082" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T12:39:16.172" grpc.time_ms=10.22 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2031vm-test-run-nixbot-gitlab> gitlab # [  215.630509] gitaly[1392]: time="2026-07-04T12:39:16.581Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=969 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7S3JSAJFY5JKVKS23Q17 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-04T12:39:46.064" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:16.564" grpc.time_ms=16.912 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2032vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=98211a6541adb8d86116a7680ad181c0d8549f29'2033vm-test-run-nixbot-gitlab> gitlab # [  215.670228] gitaly[1392]: time="2026-07-04T12:39:16.621Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7S3JSAJFY5JKVKS23Q17 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-07-04T12:39:46.106" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:16.607" grpc.time_ms=14.31 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2034vm-test-run-nixbot-gitlab> gitlab # [  215.689903] sidekiq-cluster[2150]: {"severity":"WARN","time":"2026-07-04T12:39:16.640Z","class":"CreatePipelineWorker","correlation_id":"01KWPJ7S3JSAJFY5JKVKS23Q17","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"fd1fc7ecbe895687b4904ab556cca5a4e96ea8fb","after":"98211a6541adb8d86116a7680ad181c0d8549f29","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"98211a6541adb8d86116a7680ad181c0d8549f29","ref":"refs/heads/master"},"retry":0}2035vm-test-run-nixbot-gitlab> gitlab # [  215.699848] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:16.650Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"46304db4ee24c38d9f8dfd4f","created_at":"2026-07-04T12:39:15.627Z","trace_propagation_headers":{"sentry-trace":"efaf3741cf434f84ae376dc2f2e06b7a-326a215a2e9f49d7","baggage":"sentry-trace_id=efaf3741cf434f84ae376dc2f2e06b7a,sentry-release=Unknown"},"correlation_id":"01KWPJ7S3JSAJFY5JKVKS23Q17","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:a821051b6ad56e93a2e6186c7c4663b4d9d3c5fe700cca9ac2c6212086f8e890","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:15.628Z","job_size_bytes":467,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-46304db4ee24c38d9f8dfd4f: done: 0.583445 sec","job_status":"done","queue_duration_s":0.438426,"scheduling_latency_s":0.43845,"gitaly_calls":7,"gitaly_duration_s":0.093731,"redis_calls":33,"redis_duration_s":0.008096,"redis_read_bytes":185,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.001364,"redis_cache_read_bytes":44,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001866,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003478,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001388,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":30,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":26,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.45418,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.583445,"completed_at":"2026-07-04T12:39:16.650Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.025651,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2036vm-test-run-nixbot-gitlab> gitlab # [  215.739574] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:16.690Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"6ee21b80-9228-4861-b7f8-463da714c3e3\"}"],"class":"WebHookWorker","jid":"a9954f7dd4682bba8b34af0b","created_at":"2026-07-04T12:39:15.710Z","trace_propagation_headers":{"sentry-trace":"efaf3741cf434f84ae376dc2f2e06b7a-326a215a2e9f49d7","baggage":"sentry-trace_id=efaf3741cf434f84ae376dc2f2e06b7a,sentry-release=Unknown"},"correlation_id":"01KWPJ7S3JSAJFY5JKVKS23Q17","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:054b139f2d34c9532299f432644f244af9d03d56e463671f2b26d4733f09a50f","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:15.926Z","job_size_bytes":1749,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-a9954f7dd4682bba8b34af0b: start","job_status":"start","queue_duration_s":0.763585,"scheduling_latency_s":0.763641}2037vm-test-run-nixbot-gitlab> gitlab # [  215.822645] nixbot[3144]: INFO:     ::ffff:127.0.0.1:59456 - "GET /api/repos/gitlab/root/test-flake/builds?commit=98211a6541adb8d86116a7680ad181c0d8549f29 HTTP/1.1" 200 OK2038vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=98211a6541adb8d86116a7680ad181c0d8549f29', in 0.14 seconds)2039vm-test-run-nixbot-gitlab> []2040vm-test-run-nixbot-gitlab> gitlab # [  215.841843] nixbot[3144]: INFO:     ::ffff:127.0.0.1:59472 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted2041vm-test-run-nixbot-gitlab> gitlab # [  215.862108] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:16.813Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"6ee21b80-9228-4861-b7f8-463da714c3e3\"}"],"class":"WebHookWorker","jid":"a9954f7dd4682bba8b34af0b","created_at":"2026-07-04T12:39:15.710Z","trace_propagation_headers":{"sentry-trace":"efaf3741cf434f84ae376dc2f2e06b7a-326a215a2e9f49d7","baggage":"sentry-trace_id=efaf3741cf434f84ae376dc2f2e06b7a,sentry-release=Unknown"},"correlation_id":"01KWPJ7S3JSAJFY5JKVKS23Q17","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:054b139f2d34c9532299f432644f244af9d03d56e463671f2b26d4733f09a50f","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:15.926Z","job_size_bytes":1749,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-a9954f7dd4682bba8b34af0b: done: 0.122683 sec","job_status":"done","queue_duration_s":0.763585,"scheduling_latency_s":0.763641,"redis_calls":19,"redis_duration_s":0.006149,"redis_read_bytes":31,"redis_write_bytes":5281,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00156,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000351,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3643,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002749,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001489,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.010995810999986588,"cpu_s":0.087514,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00037407,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","duration_s":0.122683,"completed_at":"2026-07-04T12:39:16.813Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003783,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2042vm-test-run-nixbot-gitlab> gitlab # [  215.870571] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:16.820Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","a9954f7dd4682bba8b34af0b"],"class":"WebHooks::LogExecutionWorker","jid":"ed52a0e04346dc309fc2bade","created_at":"2026-07-04T12:39:16.798Z","trace_propagation_headers":{"sentry-trace":"efaf3741cf434f84ae376dc2f2e06b7a-6434bb9618d44e6b","baggage":"sentry-trace_id=efaf3741cf434f84ae376dc2f2e06b7a,sentry-release=Unknown"},"correlation_id":"01KWPJ7S3JSAJFY5JKVKS23Q17","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0d6563c7cd1327c36ad9f54ad3958b7e7d146cd71525c2faf47d2e7fa8c0b7f9","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:16.805Z","job_size_bytes":2507,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-ed52a0e04346dc309fc2bade: start","job_status":"start","queue_duration_s":0.014901,"scheduling_latency_s":0.014962}2043vm-test-run-nixbot-gitlab> gitlab # [  216.014629] gitaly[1392]: time="2026-07-04T12:39:16.965Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7V2CQ4P90JEME95QFC9D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T12:39:16.955" grpc.time_ms=9.481 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc2044vm-test-run-nixbot-gitlab> gitlab # [  216.016264] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:16 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 1532045vm-test-run-nixbot-gitlab> gitlab # [  216.028095] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:16.979Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","a9954f7dd4682bba8b34af0b"],"class":"WebHooks::LogExecutionWorker","jid":"ed52a0e04346dc309fc2bade","created_at":"2026-07-04T12:39:16.798Z","trace_propagation_headers":{"sentry-trace":"efaf3741cf434f84ae376dc2f2e06b7a-6434bb9618d44e6b","baggage":"sentry-trace_id=efaf3741cf434f84ae376dc2f2e06b7a,sentry-release=Unknown"},"correlation_id":"01KWPJ7S3JSAJFY5JKVKS23Q17","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0d6563c7cd1327c36ad9f54ad3958b7e7d146cd71525c2faf47d2e7fa8c0b7f9","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:16.805Z","job_size_bytes":2507,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-ed52a0e04346dc309fc2bade: done: 0.159015 sec","job_status":"done","queue_duration_s":0.014901,"scheduling_latency_s":0.014962,"redis_calls":10,"redis_duration_s":0.006839,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002392,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003323,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001124,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.043,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.025,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.025,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.043,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.093846,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000365131,"duration_s":0.159015,"completed_at":"2026-07-04T12:39:16.979Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.048423,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2046vm-test-run-nixbot-gitlab> gitlab # [  216.218548] gitaly[1392]: time="2026-07-04T12:39:17.170Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7VA6SPQF4E2PYNXM1MRX grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T12:39:17.156" pid=1392 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc2047vm-test-run-nixbot-gitlab> gitlab # [  216.235504] gitaly[1392]: time="2026-07-04T12:39:17.170Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7VA6SPQF4E2PYNXM1MRX grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T12:39:17.156" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc2048vm-test-run-nixbot-gitlab> gitlab # [  216.236888] gitlab-workhorse[1293]: time="2026-07-04T12:39:17Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672049vm-test-run-nixbot-gitlab> gitlab # [  216.238041] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:17 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 1092050vm-test-run-nixbot-gitlab> gitlab # [  216.238953] gitaly[1392]: time="2026-07-04T12:39:17.170Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7VA6SPQF4E2PYNXM1MRX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T12:39:17.156" grpc.time_ms=14.238 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc2051vm-test-run-nixbot-gitlab> gitlab # [  216.304269] gitaly[1392]: time="2026-07-04T12:39:17.255Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7VDQCPR3VBBRM4SZWFZR grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T12:39:17.241" pid=1392 remote_ip="2001:db8:1::1" request_sha=df567c5dae29c805409d72700287fb4315de8c78 response_bytes=384 span.kind=server system=grpc2052vm-test-run-nixbot-gitlab> gitlab # [  216.305480] gitaly[1392]: time="2026-07-04T12:39:17.255Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1204 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7VDQCPR3VBBRM4SZWFZR grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T12:39:17.241" negotiation.command=fetch negotiation.haves=1 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc2053vm-test-run-nixbot-gitlab> gitlab # [  216.306265] gitlab-workhorse[1293]: time="2026-07-04T12:39:17Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3842054vm-test-run-nixbot-gitlab> gitlab # [  216.307207] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:17 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 384 "" "git/2.54.0" 802055vm-test-run-nixbot-gitlab> gitlab # [  216.307691] gitaly[1392]: time="2026-07-04T12:39:17.255Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1204 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7VDQCPR3VBBRM4SZWFZR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T12:39:17.241" grpc.time_ms=13.545 negotiation.command=fetch negotiation.haves=1 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc2056vm-test-run-nixbot-gitlab> gitlab # [  216.401488] nixbot[3144]: {"ts": "2026-07-04T12:39:17.352Z", "level": "info", "logger": "nixbot.canceller", "msg": "cancelling build", "build_id": 1, "reason": "superseded"}2057vm-test-run-nixbot-gitlab> gitlab # [  216.407888] nix-daemon[3312]: unexpected Nix daemon error: error: interrupted by the user2058vm-test-run-nixbot-gitlab> gitlab # [  216.462363] gitaly[1392]: time="2026-07-04T12:39:17.413Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7VKRGXDDXWZYTCCK5NCM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T12:39:47.011" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:17.412" grpc.time_ms=0.744 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2059vm-test-run-nixbot-gitlab> gitlab # [  216.473276] gitaly[1392]: time="2026-07-04T12:39:17.424Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7VM1JSCQ5KMAC6HZVQYX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T12:39:47.423" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:17.423" grpc.time_ms=0.701 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2060vm-test-run-nixbot-gitlab> gitlab # [  216.509362] gitaly[1392]: time="2026-07-04T12:39:17.460Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7VM1JSCQ5KMAC6HZVQYX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-04T12:39:27.163" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T12:39:17.453" grpc.time_ms=7.15 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2061vm-test-run-nixbot-gitlab> gitlab # [  216.545485] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:17.496Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"6c10f3a0dc61c1421f20f651","created_at":"2026-07-04T12:39:17.491Z","trace_propagation_headers":{"sentry-trace":"c13f9ee52977433eb49daef60bbcbb59-d38ae39b5e9b4ed9","baggage":"sentry-trace_id=c13f9ee52977433eb49daef60bbcbb59,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPJ7VM1JSCQ5KMAC6HZVQYX","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:17.495Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-6c10f3a0dc61c1421f20f651: start","job_status":"start","queue_duration_s":0.001576,"scheduling_latency_s":0.001617}2062vm-test-run-nixbot-gitlab> gitlab # [  216.553861] gitaly[1392]: time="2026-07-04T12:39:17.505Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=650 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7VKRGXDDXWZYTCCK5NCM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-07-04T12:39:47.095" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T12:39:17.495" grpc.time_ms=9.741 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2063vm-test-run-nixbot-gitlab> gitlab # [  216.568573] gitaly[1392]: time="2026-07-04T12:39:17.520Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7VKRGXDDXWZYTCCK5NCM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-04T12:39:27.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T12:39:17.514" grpc.time_ms=5.955 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2064vm-test-run-nixbot-gitlab> gitlab # [  216.613216] gitaly[1392]: time="2026-07-04T12:39:17.564Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7VKRGXDDXWZYTCCK5NCM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-04T12:39:27.053" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T12:39:17.564" grpc.time_ms=0.214 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2065vm-test-run-nixbot-gitlab> gitlab # [  216.632629] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:17 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/fd1fc7ecbe895687b4904ab556cca5a4e96ea8fb HTTP/1.1" 201 651 "" "python-httpx/0.28.1" 2062066vm-test-run-nixbot-gitlab> gitlab # [  216.637993] nixbot[3144]: {"ts": "2026-07-04T12:39:17.589Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/fd1fc7ecbe895687b4904ab556cca5a4e96ea8fb \"HTTP/1.1 201 Created\""}2067vm-test-run-nixbot-gitlab> gitlab # [  216.647818] gitaly[1392]: time="2026-07-04T12:39:17.599Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=616 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=8 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7VKRGXDDXWZYTCCK5NCM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-04T12:39:27.077" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T12:39:17.587" grpc.time_ms=11.707 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2068vm-test-run-nixbot-gitlab> gitlab # [  216.670738] gitaly[1392]: time="2026-07-04T12:39:17.622Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7VM1JSCQ5KMAC6HZVQYX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T12:39:17.621" grpc.time_ms=0.339 pid=1392 span.kind=server system=grpc2069vm-test-run-nixbot-gitlab> gitlab # [  216.697733] gitaly[1392]: time="2026-07-04T12:39:17.649Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7VM1JSCQ5KMAC6HZVQYX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T12:39:17.648" grpc.time_ms=0.319 pid=1392 span.kind=server system=grpc2070vm-test-run-nixbot-gitlab> gitlab # [  216.715654] gitaly[1392]: time="2026-07-04T12:39:17.667Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7VVJQGT4A2CDPBS2SAJE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T12:39:47.666" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:17.666" grpc.time_ms=0.602 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2071vm-test-run-nixbot-gitlab> gitlab # [  216.736250] gitaly[1392]: time="2026-07-04T12:39:17.686Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7VM1JSCQ5KMAC6HZVQYX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T12:39:17.685" grpc.time_ms=0.313 pid=1392 span.kind=server system=grpc2072vm-test-run-nixbot-gitlab> gitlab # [  216.751744] gitaly[1392]: time="2026-07-04T12:39:17.703Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7VKRGXDDXWZYTCCK5NCM grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4159 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T12:39:17.701" grpc.time_ms=1.538 pid=1392 span.kind=server system=grpc2073vm-test-run-nixbot-gitlab> gitlab # [  216.767912] gitaly[1392]: time="2026-07-04T12:39:17.715Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7VM1JSCQ5KMAC6HZVQYX diskcache=279d08bb-e7d6-44db-be7c-1609b37f10ae grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-04T12:39:47.080" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-04T12:39:17.581" pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2074vm-test-run-nixbot-gitlab> gitlab # [  216.783598] gitaly[1392]: time="2026-07-04T12:39:17.716Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=104 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=6890 command.oublock=0 command.real_time_ms=122 command.spawn_token_fork_ms=0 command.system_time_ms=78 command.user_time_ms=26 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7VM1JSCQ5KMAC6HZVQYX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-04T12:39:47.080" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T12:39:17.581" grpc.time_ms=134.497 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2075vm-test-run-nixbot-gitlab> gitlab # [  216.804798] gitaly[1392]: time="2026-07-04T12:39:17.745Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7VVJQGT4A2CDPBS2SAJE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-04T12:39:27.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T12:39:17.735" grpc.time_ms=9.763 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2076vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=98211a6541adb8d86116a7680ad181c0d8549f29'2077vm-test-run-nixbot-gitlab> gitlab # [  216.827579] gitaly[1392]: time="2026-07-04T12:39:17.757Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7VKRGXDDXWZYTCCK5NCM grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4161 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T12:39:17.757" grpc.time_ms=0.371 pid=1392 span.kind=server system=grpc2078vm-test-run-nixbot-gitlab> gitlab # [  216.830161] gitaly[1392]: time="2026-07-04T12:39:17.769Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7VKRGXDDXWZYTCCK5NCM diskcache=e038b155-ac0b-49af-bb07-739fa4212d05 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-04T12:39:27.128" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T12:39:17.639" pid=1392 ref=refs/keep-around/98211a6541adb8d86116a7680ad181c0d8549f29 remote_ip="2001:db8:1::1" revision=98211a6541adb8d86116a7680ad181c0d8549f29 span.kind=server system=grpc username=root2079vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=98211a6541adb8d86116a7680ad181c0d8549f29', in 0.04 seconds)2080vm-test-run-nixbot-gitlab> [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '98211a6541adb8d86116a7680ad181c0d8549f29', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-07-04T12:39:17.344541Z', 'started_at': None, 'finished_at': None}]2081vm-test-run-nixbot-gitlab> (finished: subtest: push delivers the webhook, in 3.30 seconds)2082vm-test-run-nixbot-gitlab> subtest: eval and build post commit statuses2083vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/98211a6541adb8d86116a7680ad181c0d8549f29/statuses2084vm-test-run-nixbot-gitlab> gitlab # [  216.831168] gitaly[1392]: time="2026-07-04T12:39:17.770Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=68 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5082 command.oublock=8 command.real_time_ms=121 command.spawn_token_fork_ms=2 command.system_time_ms=51 command.user_time_ms=17 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7VKRGXDDXWZYTCCK5NCM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-04T12:39:27.128" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T12:39:17.639" grpc.time_ms=130.975 pid=1392 ref=refs/keep-around/98211a6541adb8d86116a7680ad181c0d8549f29 remote_ip="2001:db8:1::1" revision=98211a6541adb8d86116a7680ad181c0d8549f29 span.kind=server system=grpc username=root2085vm-test-run-nixbot-gitlab> gitlab # [  216.831929] gitaly[1392]: time="2026-07-04T12:39:17.775Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7VKRGXDDXWZYTCCK5NCM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T12:39:47.374" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:17.775" grpc.time_ms=0.369 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2086vm-test-run-nixbot-gitlab> gitlab # [  216.838518] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:17 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/fd1fc7ecbe895687b4904ab556cca5a4e96ea8fb HTTP/1.1" 201 665 "" "python-httpx/0.28.1" 1712087vm-test-run-nixbot-gitlab> gitlab # [  216.840494] nixbot[3144]: {"ts": "2026-07-04T12:39:17.791Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/fd1fc7ecbe895687b4904ab556cca5a4e96ea8fb \"HTTP/1.1 201 Created\""}2088vm-test-run-nixbot-gitlab> gitlab # [  216.862271] nixbot[3144]: INFO:     ::ffff:127.0.0.1:59484 - "GET /api/repos/gitlab/root/test-flake/builds?commit=98211a6541adb8d86116a7680ad181c0d8549f29 HTTP/1.1" 200 OK2089vm-test-run-nixbot-gitlab> gitlab # [  216.941766] gitaly[1392]: time="2026-07-04T12:39:17.893Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7W2C50G9KNCMSM0G8V86 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T12:39:47.891" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:17.892" grpc.time_ms=0.622 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2090vm-test-run-nixbot-gitlab> gitlab # [  216.953343] gitaly[1392]: time="2026-07-04T12:39:17.904Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7VM1JSCQ5KMAC6HZVQYX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-04T12:39:47.403" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:17.904" grpc.time_ms=0.876 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2091vm-test-run-nixbot-gitlab> gitlab # [  216.978156] gitaly[1392]: time="2026-07-04T12:39:17.929Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7VM1JSCQ5KMAC6HZVQYX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T12:39:47.427" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:17.927" grpc.time_ms=1.073 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2092vm-test-run-nixbot-gitlab> gitlab # [  216.993497] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:17 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/98211a6541adb8d86116a7680ad181c0d8549f29 HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 5762093vm-test-run-nixbot-gitlab> gitlab # [  216.997137] nixbot[3144]: {"ts": "2026-07-04T12:39:17.947Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/98211a6541adb8d86116a7680ad181c0d8549f29 \"HTTP/1.1 201 Created\""}2094vm-test-run-nixbot-gitlab> gitlab # [  217.003996] nixbot[3144]: {"ts": "2026-07-04T12:39:17.955Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-98211a6541ad-be957e04", "argv0": "bwrap"}2095vm-test-run-nixbot-gitlab> gitlab # [  217.019812] gitaly[1392]: time="2026-07-04T12:39:17.971Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7VM1JSCQ5KMAC6HZVQYX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T12:39:47.469" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:17.970" grpc.time_ms=0.748 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2096vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/98211a6541adb8d86116a7680ad181c0d8549f29/statuses, in 0.18 seconds)2097vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': '98211a6541adb8d86116a7680ad181c0d8549f29', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-04T12:39:17.921Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2098vm-test-run-nixbot-gitlab> gitlab # [  217.040252] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:17.981Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"0c4c675b2a85671ae0ca17fe","created_at":"2026-07-04T12:39:17.978Z","trace_propagation_headers":{"sentry-trace":"c13f9ee52977433eb49daef60bbcbb59-df7f4c9987d741dd","baggage":"sentry-trace_id=c13f9ee52977433eb49daef60bbcbb59,sentry-release=Unknown"},"correlation_id":"01KWPJ7VM1JSCQ5KMAC6HZVQYX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","duplicate-of":"6c10f3a0dc61c1421f20f651","job_size_bytes":3,"pid":2150,"job_status":"deduplicated","message":"PipelineProcessWorker JID-0c4c675b2a85671ae0ca17fe: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}2099vm-test-run-nixbot-gitlab> gitlab # [  217.064650] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:17.983Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"6c10f3a0dc61c1421f20f651","created_at":"2026-07-04T12:39:17.491Z","trace_propagation_headers":{"sentry-trace":"c13f9ee52977433eb49daef60bbcbb59-d38ae39b5e9b4ed9","baggage":"sentry-trace_id=c13f9ee52977433eb49daef60bbcbb59,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPJ7VM1JSCQ5KMAC6HZVQYX","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:17.495Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2150,"job_status":"rescheduled","message":"PipelineProcessWorker JID-6c10f3a0dc61c1421f20f651: rescheduled"}2100vm-test-run-nixbot-gitlab> gitlab # [  217.088363] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:17.989Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"6c10f3a0dc61c1421f20f651","created_at":"2026-07-04T12:39:17.491Z","trace_propagation_headers":{"sentry-trace":"c13f9ee52977433eb49daef60bbcbb59-d38ae39b5e9b4ed9","baggage":"sentry-trace_id=c13f9ee52977433eb49daef60bbcbb59,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPJ7VM1JSCQ5KMAC6HZVQYX","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:17.495Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-6c10f3a0dc61c1421f20f651: done: 0.492848 sec","job_status":"done","queue_duration_s":0.001576,"scheduling_latency_s":0.001617,"gitaly_calls":4,"gitaly_duration_s":0.150754,"redis_calls":104,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.029567,"redis_read_bytes":266,"redis_write_bytes":24002,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001139,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":17,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.001852,"redis_cache_read_bytes":33,"redis_cache_write_bytes":1526,"redis_feature_flag_calls":22,"redis_feature_flag_duration_s":0.007255,"redis_feature_flag_read_bytes":22,"redis_feature_flag_write_bytes":3871,"redis_queues_calls":22,"redis_queues_duration_s":0.00487,"redis_queues_read_bytes":26,"redis_queues_write_bytes":12381,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.008923,"redis_queues_metadata_read_bytes":110,"redis_queues_metadata_write_bytes":3546,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000321,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001251,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002817,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":850,"db_count":47,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.054,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.012,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.015,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.037,"db_ci_duration_s":0.017,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.211981,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.456176693,"duration_s":0.492848,"completed_at":"2026-07-04T12:39:17.989Z","load_balancing_strategy":"primary","db_duration_s":0.059757,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2101vm-test-run-nixbot-gitlab> gitlab # [  217.161299] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:17.991Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"16c6b8ead7a4ec7a6c67ee97","created_at":"2026-07-04T12:39:17.564Z","trace_propagation_headers":{"sentry-trace":"c13f9ee52977433eb49daef60bbcbb59-df7f4c9987d741dd","baggage":"sentry-trace_id=c13f9ee52977433eb49daef60bbcbb59,sentry-release=Unknown"},"correlation_id":"01KWPJ7VM1JSCQ5KMAC6HZVQYX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:17.569Z","job_size_bytes":23,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-16c6b8ead7a4ec7a6c67ee97: start","job_status":"start","queue_duration_s":0.42149,"scheduling_latency_s":0.421514}2102vm-test-run-nixbot-gitlab> gitlab # [  217.163043] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:18.076Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"16c6b8ead7a4ec7a6c67ee97","created_at":"2026-07-04T12:39:17.564Z","trace_propagation_headers":{"sentry-trace":"c13f9ee52977433eb49daef60bbcbb59-df7f4c9987d741dd","baggage":"sentry-trace_id=c13f9ee52977433eb49daef60bbcbb59,sentry-release=Unknown"},"correlation_id":"01KWPJ7VM1JSCQ5KMAC6HZVQYX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:17.569Z","job_size_bytes":23,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-16c6b8ead7a4ec7a6c67ee97: done: 0.085317 sec","job_status":"done","queue_duration_s":0.42149,"scheduling_latency_s":0.421514,"redis_calls":15,"redis_duration_s":0.003697,"redis_read_bytes":13,"redis_write_bytes":3012,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000432,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_calls":2,"redis_queues_duration_s":0.000309,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1627,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001928,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":742,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000501,"redis_rate_limiting_write_bytes":148,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000527,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":13,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":13,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.067936,"worker_id":"sidekiq_0","rate_limiting_gates":["permanent_email_failure","temporary_email_failure"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000340546,"duration_s":0.085317,"completed_at":"2026-07-04T12:39:18.076Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010441,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2103vm-test-run-nixbot-gitlab> gitlab # [  217.204194] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:18.077Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"4a44cde5b390c1add5968a28","created_at":"2026-07-04T12:39:17.724Z","trace_propagation_headers":{"sentry-trace":"c13f9ee52977433eb49daef60bbcbb59-df7f4c9987d741dd","baggage":"sentry-trace_id=c13f9ee52977433eb49daef60bbcbb59,sentry-release=Unknown"},"correlation_id":"01KWPJ7VM1JSCQ5KMAC6HZVQYX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:17.730Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-4a44cde5b390c1add5968a28: start","job_status":"start","queue_duration_s":0.347382,"scheduling_latency_s":0.347404}2104vm-test-run-nixbot-gitlab> gitlab # [  217.230898] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:18.101Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"4a44cde5b390c1add5968a28","created_at":"2026-07-04T12:39:17.724Z","trace_propagation_headers":{"sentry-trace":"c13f9ee52977433eb49daef60bbcbb59-df7f4c9987d741dd","baggage":"sentry-trace_id=c13f9ee52977433eb49daef60bbcbb59,sentry-release=Unknown"},"correlation_id":"01KWPJ7VM1JSCQ5KMAC6HZVQYX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:17.730Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-4a44cde5b390c1add5968a28: done: 0.023809 sec","job_status":"done","queue_duration_s":0.347382,"scheduling_latency_s":0.347404,"redis_calls":10,"redis_duration_s":0.002274,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000425,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00141,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000439,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014773,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000352838,"duration_s":0.023809,"completed_at":"2026-07-04T12:39:18.101Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003135,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2105vm-test-run-nixbot-gitlab> gitlab # [  217.319381] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:18.103Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"3d373aec30c9646881016fa1","created_at":"2026-07-04T12:39:18.067Z","trace_propagation_headers":{"sentry-trace":"c13f9ee52977433eb49daef60bbcbb59-dcf402e8911f4c3f","baggage":"sentry-trace_id=c13f9ee52977433eb49daef60bbcbb59,sentry-release=Unknown"},"correlation_id":"01KWPJ7VM1JSCQ5KMAC6HZVQYX","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:b1c3bfb80154d3eeed0620cd6776884874f3dbf49a856920b7868044af74864b","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:18.070Z","job_size_bytes":464,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-3d373aec30c9646881016fa1: start","job_status":"start","queue_duration_s":0.033013,"scheduling_latency_s":0.033034}2106vm-test-run-nixbot-gitlab> gitlab # [  217.326515] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:17 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/98211a6541adb8d86116a7680ad181c0d8549f29/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1542107vm-test-run-nixbot-gitlab> gitlab # [  217.330147] sidekiq-cluster[2150]: {"severity":"WARN","time":"2026-07-04T12:39:18.143Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"3d373aec30c9646881016fa1","created_at":"2026-07-04T12:39:18.067Z","trace_propagation_headers":{"sentry-trace":"c13f9ee52977433eb49daef60bbcbb59-dcf402e8911f4c3f","baggage":"sentry-trace_id=c13f9ee52977433eb49daef60bbcbb59,sentry-release=Unknown"},"correlation_id":"01KWPJ7VM1JSCQ5KMAC6HZVQYX","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:b1c3bfb80154d3eeed0620cd6776884874f3dbf49a856920b7868044af74864b","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:18.070Z","job_size_bytes":464,"pid":2150,"sidekiq_tid":"m2x6","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-3d373aec30c9646881016fa1: fail: 0.039644 sec","job_status":"fail","queue_duration_s":0.033013,"scheduling_latency_s":0.033034,"redis_calls":7,"redis_duration_s":0.0018009999999999999,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001349,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000452,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024999,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.039644,"completed_at":"2026-07-04T12:39:18.142Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/pipelines.rb:33:in `pipeline_mail'","app/mailers/emails/pipelines.rb:10:in `pipeline_failed_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.008,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2108vm-test-run-nixbot-gitlab> gitlab # [  217.954922] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:18.145Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"a560b2d824440314c9c3cc32","created_at":"2026-07-04T12:39:17.745Z","trace_propagation_headers":{"sentry-trace":"c13f9ee52977433eb49daef60bbcbb59-df7f4c9987d741dd","baggage":"sentry-trace_id=c13f9ee52977433eb49daef60bbcbb59,sentry-release=Unknown"},"correlation_id":"01KWPJ7VM1JSCQ5KMAC6HZVQYX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:17.760Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m9wq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-a560b2d824440314c9c3cc32: start","job_status":"start","queue_duration_s":0.385369,"scheduling_latency_s":0.385412}2109vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/98211a6541adb8d86116a7680ad181c0d8549f29/statuses2110vm-test-run-nixbot-gitlab> gitlab # [  217.961202] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:18.280Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"a560b2d824440314c9c3cc32","created_at":"2026-07-04T12:39:17.745Z","trace_propagation_headers":{"sentry-trace":"c13f9ee52977433eb49daef60bbcbb59-df7f4c9987d741dd","baggage":"sentry-trace_id=c13f9ee52977433eb49daef60bbcbb59,sentry-release=Unknown"},"correlation_id":"01KWPJ7VM1JSCQ5KMAC6HZVQYX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:17.760Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m9wq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-a560b2d824440314c9c3cc32: done: 0.135171 sec","job_status":"done","queue_duration_s":0.385369,"scheduling_latency_s":0.385412,"redis_calls":10,"redis_duration_s":0.002594,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000615,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001513,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000466,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.106,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.101,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021129,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000412622,"duration_s":0.135171,"completed_at":"2026-07-04T12:39:18.280Z","load_balancing_strategy":"primary","db_duration_s":0.10621,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2111vm-test-run-nixbot-gitlab> gitlab # [  218.081531] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:18.282Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"4b3006a3ed945b57f8aca955","created_at":"2026-07-04T12:39:17.780Z","trace_propagation_headers":{"sentry-trace":"f457e1d36ac048d6ab90c7fe17318708-2d5267f4b7dd4656","baggage":"sentry-trace_id=f457e1d36ac048d6ab90c7fe17318708,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPJ7VKRGXDDXWZYTCCK5NCM","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:17.787Z","job_size_bytes":65,"pid":2150,"sidekiq_tid":"m9wq","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-4b3006a3ed945b57f8aca955: start","job_status":"start","queue_duration_s":0.494962,"scheduling_latency_s":0.494984}2112vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/98211a6541adb8d86116a7680ad181c0d8549f29/statuses, in 0.13 seconds)2113vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': '98211a6541adb8d86116a7680ad181c0d8549f29', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-04T12:39:17.921Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2114vm-test-run-nixbot-gitlab> gitlab # [  218.082904] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:18.302Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"4b3006a3ed945b57f8aca955","created_at":"2026-07-04T12:39:17.780Z","trace_propagation_headers":{"sentry-trace":"f457e1d36ac048d6ab90c7fe17318708-2d5267f4b7dd4656","baggage":"sentry-trace_id=f457e1d36ac048d6ab90c7fe17318708,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPJ7VKRGXDDXWZYTCCK5NCM","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:17.787Z","job_size_bytes":65,"pid":2150,"sidekiq_tid":"m9wq","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-4b3006a3ed945b57f8aca955: done: 0.020362 sec","job_status":"done","queue_duration_s":0.494962,"scheduling_latency_s":0.494984,"redis_calls":7,"redis_duration_s":0.0015999999999999999,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001166,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000434,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013322,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020362,"completed_at":"2026-07-04T12:39:18.302Z","load_balancing_strategy":"primary","db_duration_s":0.002459,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2115vm-test-run-nixbot-gitlab> gitlab # [  218.125615] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:18.304Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"fe3c0d025202b724534ed821","created_at":"2026-07-04T12:39:17.788Z","trace_propagation_headers":{"sentry-trace":"f457e1d36ac048d6ab90c7fe17318708-2d5267f4b7dd4656","baggage":"sentry-trace_id=f457e1d36ac048d6ab90c7fe17318708,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPJ7VKRGXDDXWZYTCCK5NCM","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:17.793Z","job_size_bytes":65,"pid":2150,"sidekiq_tid":"m9wq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-fe3c0d025202b724534ed821: start","job_status":"start","queue_duration_s":0.510412,"scheduling_latency_s":0.510434}2116vm-test-run-nixbot-gitlab> gitlab # [  218.153255] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:18.345Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"fe3c0d025202b724534ed821","created_at":"2026-07-04T12:39:17.788Z","trace_propagation_headers":{"sentry-trace":"f457e1d36ac048d6ab90c7fe17318708-2d5267f4b7dd4656","baggage":"sentry-trace_id=f457e1d36ac048d6ab90c7fe17318708,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPJ7VKRGXDDXWZYTCCK5NCM","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:17.793Z","job_size_bytes":65,"pid":2150,"sidekiq_tid":"m9wq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-fe3c0d025202b724534ed821: done: 0.041276 sec","job_status":"done","queue_duration_s":0.510412,"scheduling_latency_s":0.510434,"redis_calls":10,"redis_duration_s":0.002593,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00137,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.001223,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027107,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.041276,"completed_at":"2026-07-04T12:39:18.345Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008195,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2117vm-test-run-nixbot-gitlab> gitlab # [  218.225846] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:18.347Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"2cd71a5efc69d96b17e1c87f","created_at":"2026-07-04T12:39:17.774Z","trace_propagation_headers":{"sentry-trace":"c13f9ee52977433eb49daef60bbcbb59-df7f4c9987d741dd","baggage":"sentry-trace_id=c13f9ee52977433eb49daef60bbcbb59,sentry-release=Unknown"},"correlation_id":"01KWPJ7VM1JSCQ5KMAC6HZVQYX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:17.870Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m9wq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-2cd71a5efc69d96b17e1c87f: start","job_status":"start","queue_duration_s":0.47638,"scheduling_latency_s":0.476409}2118vm-test-run-nixbot-gitlab> gitlab # [  218.309206] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:18.375Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"2cd71a5efc69d96b17e1c87f","created_at":"2026-07-04T12:39:17.774Z","trace_propagation_headers":{"sentry-trace":"c13f9ee52977433eb49daef60bbcbb59-df7f4c9987d741dd","baggage":"sentry-trace_id=c13f9ee52977433eb49daef60bbcbb59,sentry-release=Unknown"},"correlation_id":"01KWPJ7VM1JSCQ5KMAC6HZVQYX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:17.870Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m9wq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-2cd71a5efc69d96b17e1c87f: done: 0.028406 sec","job_status":"done","queue_duration_s":0.47638,"scheduling_latency_s":0.476409,"redis_calls":13,"redis_duration_s":0.002891,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000431,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00131,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000671,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000479,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01791,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000353956,"duration_s":0.028406,"completed_at":"2026-07-04T12:39:18.375Z","load_balancing_strategy":"primary","db_duration_s":0.004074,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2119vm-test-run-nixbot-gitlab> gitlab # [  218.317188] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:18.376Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"09a4847e2259959633e4aecd","created_at":"2026-07-04T12:39:17.871Z","trace_propagation_headers":{"sentry-trace":"c13f9ee52977433eb49daef60bbcbb59-df7f4c9987d741dd","baggage":"sentry-trace_id=c13f9ee52977433eb49daef60bbcbb59,sentry-release=Unknown"},"correlation_id":"01KWPJ7VM1JSCQ5KMAC6HZVQYX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:17.878Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m9wq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-09a4847e2259959633e4aecd: start","job_status":"start","queue_duration_s":0.498118,"scheduling_latency_s":0.498138}2120vm-test-run-nixbot-gitlab> gitlab # [  218.344291] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:18.399Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"09a4847e2259959633e4aecd","created_at":"2026-07-04T12:39:17.871Z","trace_propagation_headers":{"sentry-trace":"c13f9ee52977433eb49daef60bbcbb59-df7f4c9987d741dd","baggage":"sentry-trace_id=c13f9ee52977433eb49daef60bbcbb59,sentry-release=Unknown"},"correlation_id":"01KWPJ7VM1JSCQ5KMAC6HZVQYX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:17.878Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m9wq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-09a4847e2259959633e4aecd: done: 0.023119 sec","job_status":"done","queue_duration_s":0.498118,"scheduling_latency_s":0.498138,"redis_calls":10,"redis_duration_s":0.002361,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000522,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.0014,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000439,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014091,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000381613,"duration_s":0.023119,"completed_at":"2026-07-04T12:39:18.399Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003341,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2121vm-test-run-nixbot-gitlab> gitlab # [  218.432181] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:18.401Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"6bef33240fcad18a99609827","created_at":"2026-07-04T12:39:17.879Z","trace_propagation_headers":{"sentry-trace":"c13f9ee52977433eb49daef60bbcbb59-df7f4c9987d741dd","baggage":"sentry-trace_id=c13f9ee52977433eb49daef60bbcbb59,sentry-release=Unknown"},"correlation_id":"01KWPJ7VM1JSCQ5KMAC6HZVQYX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:17.884Z","job_size_bytes":19,"pid":2150,"sidekiq_tid":"m9wq","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-6bef33240fcad18a99609827: start","job_status":"start","queue_duration_s":0.516296,"scheduling_latency_s":0.516316}2122vm-test-run-nixbot-gitlab> gitlab # [  218.440077] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:18.426Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"6bef33240fcad18a99609827","created_at":"2026-07-04T12:39:17.879Z","trace_propagation_headers":{"sentry-trace":"c13f9ee52977433eb49daef60bbcbb59-df7f4c9987d741dd","baggage":"sentry-trace_id=c13f9ee52977433eb49daef60bbcbb59,sentry-release=Unknown"},"correlation_id":"01KWPJ7VM1JSCQ5KMAC6HZVQYX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:17.884Z","job_size_bytes":19,"pid":2150,"sidekiq_tid":"m9wq","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-6bef33240fcad18a99609827: done: 0.025569 sec","job_status":"done","queue_duration_s":0.516296,"scheduling_latency_s":0.516316,"redis_calls":11,"redis_duration_s":0.002711,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000416,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001716,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000579,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016304,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000674667,"duration_s":0.025569,"completed_at":"2026-07-04T12:39:18.426Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002992,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2123vm-test-run-nixbot-gitlab> gitlab # [  218.511926] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:18.428Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"770d1708c7ef176be8d60c59","created_at":"2026-07-04T12:39:17.887Z","trace_propagation_headers":{"sentry-trace":"c13f9ee52977433eb49daef60bbcbb59-df7f4c9987d741dd","baggage":"sentry-trace_id=c13f9ee52977433eb49daef60bbcbb59,sentry-release=Unknown"},"correlation_id":"01KWPJ7VM1JSCQ5KMAC6HZVQYX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:17.894Z","job_size_bytes":65,"pid":2150,"sidekiq_tid":"m9wq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-770d1708c7ef176be8d60c59: start","job_status":"start","queue_duration_s":0.533662,"scheduling_latency_s":0.533685}2124vm-test-run-nixbot-gitlab> gitlab # [  218.539758] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:18.453Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"770d1708c7ef176be8d60c59","created_at":"2026-07-04T12:39:17.887Z","trace_propagation_headers":{"sentry-trace":"c13f9ee52977433eb49daef60bbcbb59-df7f4c9987d741dd","baggage":"sentry-trace_id=c13f9ee52977433eb49daef60bbcbb59,sentry-release=Unknown"},"correlation_id":"01KWPJ7VM1JSCQ5KMAC6HZVQYX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:17.894Z","job_size_bytes":65,"pid":2150,"sidekiq_tid":"m9wq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-770d1708c7ef176be8d60c59: done: 0.024851 sec","job_status":"done","queue_duration_s":0.533662,"scheduling_latency_s":0.533685,"redis_calls":10,"redis_duration_s":0.002297,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000458,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00137,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000469,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015241,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000350324,"duration_s":0.024851,"completed_at":"2026-07-04T12:39:18.453Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003534,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2125vm-test-run-nixbot-gitlab> gitlab # [  218.639054] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:18.454Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"94d2bddcf570a40be85e0c0e","created_at":"2026-07-04T12:39:17.895Z","trace_propagation_headers":{"sentry-trace":"c13f9ee52977433eb49daef60bbcbb59-df7f4c9987d741dd","baggage":"sentry-trace_id=c13f9ee52977433eb49daef60bbcbb59,sentry-release=Unknown"},"correlation_id":"01KWPJ7VM1JSCQ5KMAC6HZVQYX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:17.897Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m9wq","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-94d2bddcf570a40be85e0c0e: start","job_status":"start","queue_duration_s":0.557226,"scheduling_latency_s":0.557246}2126vm-test-run-nixbot-gitlab> gitlab # [  218.639694] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:18.479Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"94d2bddcf570a40be85e0c0e","created_at":"2026-07-04T12:39:17.895Z","trace_propagation_headers":{"sentry-trace":"c13f9ee52977433eb49daef60bbcbb59-df7f4c9987d741dd","baggage":"sentry-trace_id=c13f9ee52977433eb49daef60bbcbb59,sentry-release=Unknown"},"correlation_id":"01KWPJ7VM1JSCQ5KMAC6HZVQYX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:17.897Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m9wq","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-94d2bddcf570a40be85e0c0e: done: 0.024346 sec","job_status":"done","queue_duration_s":0.557226,"scheduling_latency_s":0.557246,"redis_calls":7,"redis_duration_s":0.002174,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001693,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000481,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016979,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024346,"completed_at":"2026-07-04T12:39:18.478Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002815,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2127vm-test-run-nixbot-gitlab> gitlab # [  218.705189] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:18.480Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"2106c1e25db0c79c9449d048","created_at":"2026-07-04T12:39:17.928Z","trace_propagation_headers":{"sentry-trace":"f457e1d36ac048d6ab90c7fe17318708-2d5267f4b7dd4656","baggage":"sentry-trace_id=f457e1d36ac048d6ab90c7fe17318708,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPJ7VKRGXDDXWZYTCCK5NCM","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:17.932Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m9wq","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-2106c1e25db0c79c9449d048: start","job_status":"start","queue_duration_s":0.547666,"scheduling_latency_s":0.547697}2128vm-test-run-nixbot-gitlab> gitlab # [  218.731327] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:18.595Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"2106c1e25db0c79c9449d048","created_at":"2026-07-04T12:39:17.928Z","trace_propagation_headers":{"sentry-trace":"f457e1d36ac048d6ab90c7fe17318708-2d5267f4b7dd4656","baggage":"sentry-trace_id=f457e1d36ac048d6ab90c7fe17318708,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPJ7VKRGXDDXWZYTCCK5NCM","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:17.932Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m9wq","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-2106c1e25db0c79c9449d048: done: 0.11489 sec","job_status":"done","queue_duration_s":0.547666,"scheduling_latency_s":0.547697,"gitaly_calls":3,"gitaly_duration_s":0.009529,"redis_calls":44,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.007785,"redis_read_bytes":98,"redis_write_bytes":6014,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001033,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.001886,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":2,"redis_queues_duration_s":0.000304,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001623,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000291,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000618,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000997,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":31,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":31,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":19,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.079781,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.096133167,"duration_s":0.11489,"completed_at":"2026-07-04T12:39:18.595Z","load_balancing_strategy":"primary","db_duration_s":0.024712,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2129vm-test-run-nixbot-gitlab> gitlab # [  218.824280] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:18.596Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"3114344ee6feff1f9da0e3d9","created_at":"2026-07-04T12:39:17.931Z","trace_propagation_headers":{"sentry-trace":"c13f9ee52977433eb49daef60bbcbb59-df7f4c9987d741dd","baggage":"sentry-trace_id=c13f9ee52977433eb49daef60bbcbb59,sentry-release=Unknown"},"correlation_id":"01KWPJ7VM1JSCQ5KMAC6HZVQYX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:17.935Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m9wq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-3114344ee6feff1f9da0e3d9: start","job_status":"start","queue_duration_s":0.661733,"scheduling_latency_s":0.661755}2130vm-test-run-nixbot-gitlab> gitlab # [  218.838219] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:18.625Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"3114344ee6feff1f9da0e3d9","created_at":"2026-07-04T12:39:17.931Z","trace_propagation_headers":{"sentry-trace":"c13f9ee52977433eb49daef60bbcbb59-df7f4c9987d741dd","baggage":"sentry-trace_id=c13f9ee52977433eb49daef60bbcbb59,sentry-release=Unknown"},"correlation_id":"01KWPJ7VM1JSCQ5KMAC6HZVQYX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:17.935Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m9wq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-3114344ee6feff1f9da0e3d9: done: 0.028673 sec","job_status":"done","queue_duration_s":0.661733,"scheduling_latency_s":0.661755,"redis_calls":10,"redis_duration_s":0.002368,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000458,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001423,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000487,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015833,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000350324,"duration_s":0.028673,"completed_at":"2026-07-04T12:39:18.625Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007302,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2131vm-test-run-nixbot-gitlab> gitlab # [  218.968923] gitaly[1392]: time="2026-07-04T12:39:18.532Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7VKRGXDDXWZYTCCK5NCM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-04T12:39:48.030" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:18.531" grpc.time_ms=0.925 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2132vm-test-run-nixbot-gitlab> gitlab # [  218.970460] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:19 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/98211a6541adb8d86116a7680ad181c0d8549f29/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 982133vm-test-run-nixbot-gitlab> gitlab # [  218.971332] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:18.626Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"758a137e35670acf2c90dcc1","created_at":"2026-07-04T12:39:17.935Z","trace_propagation_headers":{"sentry-trace":"c13f9ee52977433eb49daef60bbcbb59-df7f4c9987d741dd","baggage":"sentry-trace_id=c13f9ee52977433eb49daef60bbcbb59,sentry-release=Unknown"},"correlation_id":"01KWPJ7VM1JSCQ5KMAC6HZVQYX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:17.938Z","job_size_bytes":6,"pid":2150,"sidekiq_tid":"m9wq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-758a137e35670acf2c90dcc1: start","job_status":"start","queue_duration_s":0.688208,"scheduling_latency_s":0.688229}2134vm-test-run-nixbot-gitlab> gitlab # [  218.972188] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:18.645Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"758a137e35670acf2c90dcc1","created_at":"2026-07-04T12:39:17.935Z","trace_propagation_headers":{"sentry-trace":"c13f9ee52977433eb49daef60bbcbb59-df7f4c9987d741dd","baggage":"sentry-trace_id=c13f9ee52977433eb49daef60bbcbb59,sentry-release=Unknown"},"correlation_id":"01KWPJ7VM1JSCQ5KMAC6HZVQYX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:17.938Z","job_size_bytes":6,"pid":2150,"sidekiq_tid":"m9wq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-758a137e35670acf2c90dcc1: done: 0.018962 sec","job_status":"done","queue_duration_s":0.688208,"scheduling_latency_s":0.688229,"redis_calls":10,"redis_duration_s":0.002249,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00044,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001346,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000463,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011496,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034334,"duration_s":0.018962,"completed_at":"2026-07-04T12:39:18.645Z","load_balancing_strategy":"primary","db_duration_s":0.001699,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2135vm-test-run-nixbot-gitlab> gitlab # [  219.059322] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:18.647Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"50b497e13c9e92d1e501d8df","created_at":"2026-07-04T12:39:17.983Z","trace_propagation_headers":{"sentry-trace":"c13f9ee52977433eb49daef60bbcbb59-df7f4c9987d741dd","baggage":"sentry-trace_id=c13f9ee52977433eb49daef60bbcbb59,sentry-release=Unknown"},"correlation_id":"01KWPJ7VM1JSCQ5KMAC6HZVQYX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:17.984Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m9wq","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-50b497e13c9e92d1e501d8df: start","job_status":"start","queue_duration_s":0.662657,"scheduling_latency_s":0.662677}2136vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/98211a6541adb8d86116a7680ad181c0d8549f29/statuses2137vm-test-run-nixbot-gitlab> gitlab # [  219.060805] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:18.713Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"50b497e13c9e92d1e501d8df","created_at":"2026-07-04T12:39:17.983Z","trace_propagation_headers":{"sentry-trace":"c13f9ee52977433eb49daef60bbcbb59-df7f4c9987d741dd","baggage":"sentry-trace_id=c13f9ee52977433eb49daef60bbcbb59,sentry-release=Unknown"},"correlation_id":"01KWPJ7VM1JSCQ5KMAC6HZVQYX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:17.984Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m9wq","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-50b497e13c9e92d1e501d8df: done: 0.066518 sec","job_status":"done","queue_duration_s":0.662657,"scheduling_latency_s":0.662677,"gitaly_calls":2,"gitaly_duration_s":0.006473,"redis_calls":19,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.003239,"redis_read_bytes":42,"redis_write_bytes":1891,"redis_cache_calls":8,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000637,"redis_cache_read_bytes":16,"redis_cache_write_bytes":735,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001359,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000259,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000984,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":20,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":20,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":9,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.045694,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.046649225,"duration_s":0.066518,"completed_at":"2026-07-04T12:39:18.713Z","load_balancing_strategy":"primary","db_duration_s":0.012719,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2138vm-test-run-nixbot-gitlab> gitlab # [  219.127886] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:18.715Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"8b1775949aad1aa61a733b5d","created_at":"2026-07-04T12:39:18.520Z","trace_propagation_headers":{"sentry-trace":"f457e1d36ac048d6ab90c7fe17318708-4288550b64774fb2","baggage":"sentry-trace_id=f457e1d36ac048d6ab90c7fe17318708,sentry-release=Unknown"},"correlation_id":"01KWPJ7VKRGXDDXWZYTCCK5NCM","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:18.522Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m9wq","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-8b1775949aad1aa61a733b5d: start","job_status":"start","queue_duration_s":0.192728,"scheduling_latency_s":0.19275}2139vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/98211a6541adb8d86116a7680ad181c0d8549f29/statuses, in 0.10 seconds)2140vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': '98211a6541adb8d86116a7680ad181c0d8549f29', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-04T12:39:17.921Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2141vm-test-run-nixbot-gitlab> gitlab # [  219.153810] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:18.738Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"8b1775949aad1aa61a733b5d","created_at":"2026-07-04T12:39:18.520Z","trace_propagation_headers":{"sentry-trace":"f457e1d36ac048d6ab90c7fe17318708-4288550b64774fb2","baggage":"sentry-trace_id=f457e1d36ac048d6ab90c7fe17318708,sentry-release=Unknown"},"correlation_id":"01KWPJ7VKRGXDDXWZYTCCK5NCM","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:18.522Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"m9wq","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-8b1775949aad1aa61a733b5d: done: 0.02378 sec","job_status":"done","queue_duration_s":0.192728,"scheduling_latency_s":0.19275,"redis_calls":7,"redis_duration_s":0.001719,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00124,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000479,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01661,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02378,"completed_at":"2026-07-04T12:39:18.738Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002745,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2142vm-test-run-nixbot-gitlab> gitlab # [  219.257390] gitaly[1392]: time="2026-07-04T12:39:18.557Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7VKRGXDDXWZYTCCK5NCM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T12:39:48.056" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:18.556" grpc.time_ms=0.712 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2143vm-test-run-nixbot-gitlab> gitlab # [  219.259074] gitaly[1392]: time="2026-07-04T12:39:18.586Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7VKRGXDDXWZYTCCK5NCM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T12:39:48.084" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:18.585" grpc.time_ms=0.717 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2144vm-test-run-nixbot-gitlab> gitlab # [  219.260573] gitaly[1392]: time="2026-07-04T12:39:18.679Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ7VM1JSCQ5KMAC6HZVQYX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-04T12:39:48.177" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:18.678" grpc.time_ms=1.004 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2145vm-test-run-nixbot-gitlab> gitlab # [  219.341275] gitaly[1392]: time="2026-07-04T12:39:18.704Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7VM1JSCQ5KMAC6HZVQYX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T12:39:48.203" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:18.703" grpc.time_ms=0.776 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2146vm-test-run-nixbot-gitlab> gitlab # [  219.342856] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:20 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/98211a6541adb8d86116a7680ad181c0d8549f29/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 682147vm-test-run-nixbot-gitlab> gitlab # [  219.343950] gitaly[1392]: time="2026-07-04T12:39:19.073Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7X7NRH5VZNA6R3FCB7X1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T12:39:49.070" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:19.071" grpc.time_ms=2.023 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2148vm-test-run-nixbot-gitlab> gitlab # [  219.345139] gitaly[1392]: time="2026-07-04T12:39:20.215Z" level=info msg="finished unary call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7YB4V9KEH1CYSK67VKSQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T12:39:50.207" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:20.208" grpc.time_ms=2.715 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2149vm-test-run-nixbot-gitlab> gitlab # [  219.476042] nix-daemon[3304]: accepted connection from pid 3547, user nixbot2150vm-test-run-nixbot-gitlab> gitlab # [  219.478889] nix-daemon[3304]: accepted connection from pid 3548, user nixbot2151vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/98211a6541adb8d86116a7680ad181c0d8549f29/statuses2152vm-test-run-nixbot-gitlab> gitlab # [  220.406695] gitaly[1392]: time="2026-07-04T12:39:21.357Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ7ZE40XF24YF24B220XVN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T12:39:51.354" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:21.355" grpc.time_ms=1.306 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2153vm-test-run-nixbot-gitlab> gitlab # [  220.479429] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:21 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/98211a6541adb8d86116a7680ad181c0d8549f29/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1452154vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/98211a6541adb8d86116a7680ad181c0d8549f29/statuses, in 0.21 seconds)2155vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': '98211a6541adb8d86116a7680ad181c0d8549f29', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-04T12:39:17.921Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2156vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/98211a6541adb8d86116a7680ad181c0d8549f29/statuses2157vm-test-run-nixbot-gitlab> gitlab # [  221.618687] gitaly[1392]: time="2026-07-04T12:39:22.569Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ80KX5ZB8G9S7PADXDY2J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T12:39:52.167" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:22.567" grpc.time_ms=1.018 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2158vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/98211a6541adb8d86116a7680ad181c0d8549f29/statuses, in 0.17 seconds)2159vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': '98211a6541adb8d86116a7680ad181c0d8549f29', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-04T12:39:17.921Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2160vm-test-run-nixbot-gitlab> gitlab # [  221.667329] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:22 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/98211a6541adb8d86116a7680ad181c0d8549f29/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1042161vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/98211a6541adb8d86116a7680ad181c0d8549f29/statuses2162vm-test-run-nixbot-gitlab> gitlab # [  222.791790] gitaly[1392]: time="2026-07-04T12:39:23.742Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ81RHTW88S3QWNEXJATAY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T12:39:53.339" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:23.740" grpc.time_ms=1.323 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2163vm-test-run-nixbot-gitlab> gitlab # [  222.813826] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:23 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/98211a6541adb8d86116a7680ad181c0d8549f29/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 982164vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/98211a6541adb8d86116a7680ad181c0d8549f29/statuses, in 0.18 seconds)2165vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': '98211a6541adb8d86116a7680ad181c0d8549f29', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-04T12:39:17.921Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2166vm-test-run-nixbot-gitlab> gitlab # [  223.446396] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:24.396Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"a2c8e5b697d43ffa4895cd32","created_at":"2026-07-04T12:39:01.074Z","trace_propagation_headers":{"sentry-trace":"fb2e26a1c2c1491ca1cd6a9a14feabf5-e08489fc81db45ca","baggage":"sentry-trace_id=fb2e26a1c2c1491ca1cd6a9a14feabf5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPJ7BHJMDWXGMDQWB8ERJJ1","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:08963546953dca552c18883018e9cd97e4b7e06fd1f97c2d449a9856196ebe14","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:24.392Z","retry_count":0,"job_size_bytes":451,"pid":2150,"sidekiq_tid":"m9wq","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-a2c8e5b697d43ffa4895cd32: start","job_status":"start","queue_duration_s":0.00335,"scheduling_latency_s":0.003417}2167vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/98211a6541adb8d86116a7680ad181c0d8549f29/statuses2168vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/98211a6541adb8d86116a7680ad181c0d8549f29/statuses, in 0.27 seconds)2169vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': '98211a6541adb8d86116a7680ad181c0d8549f29', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-04T12:39:17.921Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2170vm-test-run-nixbot-gitlab> gitlab # [  223.586228] sidekiq-cluster[2150]: {"severity":"WARN","time":"2026-07-04T12:39:24.536Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"a2c8e5b697d43ffa4895cd32","created_at":"2026-07-04T12:39:01.074Z","trace_propagation_headers":{"sentry-trace":"fb2e26a1c2c1491ca1cd6a9a14feabf5-e08489fc81db45ca","baggage":"sentry-trace_id=fb2e26a1c2c1491ca1cd6a9a14feabf5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPJ7BHJMDWXGMDQWB8ERJJ1","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:08963546953dca552c18883018e9cd97e4b7e06fd1f97c2d449a9856196ebe14","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:24.392Z","retry_count":0,"job_size_bytes":451,"pid":2150,"sidekiq_tid":"m9wq","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-a2c8e5b697d43ffa4895cd32: fail: 0.140505 sec","job_status":"fail","queue_duration_s":0.00335,"scheduling_latency_s":0.003417,"redis_calls":7,"redis_duration_s":0.004097,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003019,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001078,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.050244,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.140505,"completed_at":"2026-07-04T12:39:24.536Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.009033,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2171vm-test-run-nixbot-gitlab> gitlab # [  224.153883] gitaly[1392]: time="2026-07-04T12:39:24.934Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ82YY77TN318TXAAQC2V8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T12:39:54.532" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:24.932" grpc.time_ms=1.421 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2172vm-test-run-nixbot-gitlab> gitlab # [  224.238398] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:24 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/98211a6541adb8d86116a7680ad181c0d8549f29/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 572173vm-test-run-nixbot-gitlab> gitlab # [  224.296712] nix-daemon[3304]: accepted connection from pid 3581, user nixbot2174vm-test-run-nixbot-gitlab> gitlab # [  224.330103] gitaly[1392]: time="2026-07-04T12:39:25.281Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ839N05PX7EG89Y4MK6K4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T12:39:55.280" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:25.280" grpc.time_ms=0.578 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2175vm-test-run-nixbot-gitlab> gitlab # [  224.393344] gitaly[1392]: time="2026-07-04T12:39:25.344Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ839N05PX7EG89Y4MK6K4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-04T12:39:35.043" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T12:39:25.333" grpc.time_ms=10.829 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2176vm-test-run-nixbot-gitlab> gitlab # [  224.432494] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:25.383Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"ee7621a8f3b2e2738744a4d9","created_at":"2026-07-04T12:39:25.372Z","trace_propagation_headers":{"sentry-trace":"9c330060dd2f4ac98e5b541dc55c8d51-96c094e3d16f425f","baggage":"sentry-trace_id=9c330060dd2f4ac98e5b541dc55c8d51,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPJ839N05PX7EG89Y4MK6K4","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:25.380Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-ee7621a8f3b2e2738744a4d9: start","job_status":"start","queue_duration_s":0.00314,"scheduling_latency_s":0.003184}2177vm-test-run-nixbot-gitlab> gitlab # [  224.481848] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:25 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/98211a6541adb8d86116a7680ad181c0d8549f29 HTTP/1.1" 201 669 "" "python-httpx/0.28.1" 1952178vm-test-run-nixbot-gitlab> gitlab # [  224.484478] nixbot[3144]: {"ts": "2026-07-04T12:39:25.434Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/98211a6541adb8d86116a7680ad181c0d8549f29 \"HTTP/1.1 201 Created\""}2179vm-test-run-nixbot-gitlab> gitlab # [  224.550260] gitaly[1392]: time="2026-07-04T12:39:25.501Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ83FXP2AGD76Z91XKQR1K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T12:39:55.099" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:25.500" grpc.time_ms=1.277 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2180vm-test-run-nixbot-gitlab> gitlab # [  224.577579] gitaly[1392]: time="2026-07-04T12:39:25.529Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPJ839N05PX7EG89Y4MK6K4 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T12:39:25.528" grpc.time_ms=0.64 pid=1392 span.kind=server system=grpc2181vm-test-run-nixbot-gitlab> gitlab # [  224.588832] gitaly[1392]: time="2026-07-04T12:39:25.540Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=3 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ83FXP2AGD76Z91XKQR1K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-04T12:39:35.016" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T12:39:25.526" grpc.time_ms=13.288 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2182vm-test-run-nixbot-gitlab> gitlab # [  224.632897] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:25 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/98211a6541adb8d86116a7680ad181c0d8549f29 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1462183vm-test-run-nixbot-gitlab> gitlab # [  224.635339] nixbot[3144]: {"ts": "2026-07-04T12:39:25.585Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/98211a6541adb8d86116a7680ad181c0d8549f29 \"HTTP/1.1 201 Created\""}2184vm-test-run-nixbot-gitlab> gitlab # [  224.639576] gitaly[1392]: time="2026-07-04T12:39:25.591Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPJ839N05PX7EG89Y4MK6K4 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T12:39:25.590" grpc.time_ms=0.827 pid=1392 span.kind=server system=grpc2185vm-test-run-nixbot-gitlab> gitlab # [  224.699379] gitaly[1392]: time="2026-07-04T12:39:25.650Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPJ839N05PX7EG89Y4MK6K4 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T12:39:25.649" grpc.time_ms=0.745 pid=1392 span.kind=server system=grpc2186vm-test-run-nixbot-gitlab> gitlab # [  224.727426] gitaly[1392]: time="2026-07-04T12:39:25.678Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ839N05PX7EG89Y4MK6K4 diskcache=c4bbfbc6-4574-4b4b-8e7f-be30190a4c8d grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-04T12:39:55.466" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-04T12:39:25.467" pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2187vm-test-run-nixbot-gitlab> gitlab # [  224.729268] gitaly[1392]: time="2026-07-04T12:39:25.679Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=112 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=6914 command.oublock=0 command.real_time_ms=200 command.spawn_token_fork_ms=1 command.system_time_ms=68 command.user_time_ms=44 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ839N05PX7EG89Y4MK6K4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-04T12:39:55.466" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T12:39:25.467" grpc.time_ms=211.363 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2188vm-test-run-nixbot-gitlab> gitlab # [  224.845513] gitaly[1392]: time="2026-07-04T12:39:25.796Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ839N05PX7EG89Y4MK6K4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-04T12:39:55.294" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:25.795" grpc.time_ms=1.351 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2189vm-test-run-nixbot-gitlab> gitlab # [  224.901647] gitaly[1392]: time="2026-07-04T12:39:25.853Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ839N05PX7EG89Y4MK6K4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T12:39:55.350" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:25.851" grpc.time_ms=1.285 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2190vm-test-run-nixbot-gitlab> gitlab # [  224.958943] gitaly[1392]: time="2026-07-04T12:39:25.910Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ839N05PX7EG89Y4MK6K4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T12:39:55.407" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:25.908" grpc.time_ms=1.308 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2191vm-test-run-nixbot-gitlab> gitlab # [  224.987704] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:25.920Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"75f5ee744d0c70785befa088","created_at":"2026-07-04T12:39:25.917Z","trace_propagation_headers":{"sentry-trace":"9c330060dd2f4ac98e5b541dc55c8d51-592ca9bb08e84302","baggage":"sentry-trace_id=9c330060dd2f4ac98e5b541dc55c8d51,sentry-release=Unknown"},"correlation_id":"01KWPJ839N05PX7EG89Y4MK6K4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"ee7621a8f3b2e2738744a4d9","job_size_bytes":3,"pid":2150,"job_status":"deduplicated","message":"PipelineProcessWorker JID-75f5ee744d0c70785befa088: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}2192vm-test-run-nixbot-gitlab> gitlab # [  224.988742] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:25.922Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"ee7621a8f3b2e2738744a4d9","created_at":"2026-07-04T12:39:25.372Z","trace_propagation_headers":{"sentry-trace":"9c330060dd2f4ac98e5b541dc55c8d51-96c094e3d16f425f","baggage":"sentry-trace_id=9c330060dd2f4ac98e5b541dc55c8d51,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPJ839N05PX7EG89Y4MK6K4","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:25.380Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2150,"job_status":"rescheduled","message":"PipelineProcessWorker JID-ee7621a8f3b2e2738744a4d9: rescheduled"}2193vm-test-run-nixbot-gitlab> gitlab # [  224.989878] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:25.929Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"ee7621a8f3b2e2738744a4d9","created_at":"2026-07-04T12:39:25.372Z","trace_propagation_headers":{"sentry-trace":"9c330060dd2f4ac98e5b541dc55c8d51-96c094e3d16f425f","baggage":"sentry-trace_id=9c330060dd2f4ac98e5b541dc55c8d51,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPJ839N05PX7EG89Y4MK6K4","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:25.380Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-ee7621a8f3b2e2738744a4d9: done: 0.545738 sec","job_status":"done","queue_duration_s":0.00314,"scheduling_latency_s":0.003184,"gitaly_calls":4,"gitaly_duration_s":0.267511,"redis_calls":93,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.057927,"redis_read_bytes":262,"redis_write_bytes":23388,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.010945,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":21,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.004931,"redis_cache_read_bytes":39,"redis_cache_write_bytes":1840,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00235,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":712,"redis_queues_calls":25,"redis_queues_duration_s":0.009442,"redis_queues_read_bytes":28,"redis_queues_write_bytes":14469,"redis_queues_metadata_calls":23,"redis_queues_metadata_duration_s":0.012551,"redis_queues_metadata_read_bytes":112,"redis_queues_metadata_write_bytes":3754,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000383,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001763,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.004617,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":38,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":38,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.056,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":23,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.01,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.012,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.033,"db_ci_duration_s":0.023,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.183518,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.511012154,"duration_s":0.545738,"completed_at":"2026-07-04T12:39:25.928Z","load_balancing_strategy":"primary","db_duration_s":0.059453,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2194vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/98211a6541adb8d86116a7680ad181c0d8549f29/statuses2195vm-test-run-nixbot-gitlab> gitlab # [  225.059454] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:25.931Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"f2a012a7b6add36ee5fbcfd8","created_at":"2026-07-04T12:39:25.459Z","trace_propagation_headers":{"sentry-trace":"9c330060dd2f4ac98e5b541dc55c8d51-592ca9bb08e84302","baggage":"sentry-trace_id=9c330060dd2f4ac98e5b541dc55c8d51,sentry-release=Unknown"},"correlation_id":"01KWPJ839N05PX7EG89Y4MK6K4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:25.461Z","job_size_bytes":23,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-f2a012a7b6add36ee5fbcfd8: start","job_status":"start","queue_duration_s":0.469804,"scheduling_latency_s":0.469828}2196vm-test-run-nixbot-gitlab> gitlab # [  225.074271] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:25.994Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"f2a012a7b6add36ee5fbcfd8","created_at":"2026-07-04T12:39:25.459Z","trace_propagation_headers":{"sentry-trace":"9c330060dd2f4ac98e5b541dc55c8d51-592ca9bb08e84302","baggage":"sentry-trace_id=9c330060dd2f4ac98e5b541dc55c8d51,sentry-release=Unknown"},"correlation_id":"01KWPJ839N05PX7EG89Y4MK6K4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:25.461Z","job_size_bytes":23,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-f2a012a7b6add36ee5fbcfd8: done: 0.062886 sec","job_status":"done","queue_duration_s":0.469804,"scheduling_latency_s":0.469828,"redis_calls":7,"redis_duration_s":0.003919,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002959,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00096,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.046866,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.062886,"completed_at":"2026-07-04T12:39:25.994Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007603,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2197vm-test-run-nixbot-gitlab> gitlab # [  225.189074] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:25.997Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"7a61c2f9488e6fefbf94c42f","created_at":"2026-07-04T12:39:25.724Z","trace_propagation_headers":{"sentry-trace":"9c330060dd2f4ac98e5b541dc55c8d51-592ca9bb08e84302","baggage":"sentry-trace_id=9c330060dd2f4ac98e5b541dc55c8d51,sentry-release=Unknown"},"correlation_id":"01KWPJ839N05PX7EG89Y4MK6K4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:25.729Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-7a61c2f9488e6fefbf94c42f: start","job_status":"start","queue_duration_s":0.267854,"scheduling_latency_s":0.267902}2198vm-test-run-nixbot-gitlab> gitlab # [  225.190454] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:26.035Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"7a61c2f9488e6fefbf94c42f","created_at":"2026-07-04T12:39:25.724Z","trace_propagation_headers":{"sentry-trace":"9c330060dd2f4ac98e5b541dc55c8d51-592ca9bb08e84302","baggage":"sentry-trace_id=9c330060dd2f4ac98e5b541dc55c8d51,sentry-release=Unknown"},"correlation_id":"01KWPJ839N05PX7EG89Y4MK6K4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:25.729Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-7a61c2f9488e6fefbf94c42f: done: 0.037917 sec","job_status":"done","queue_duration_s":0.267854,"scheduling_latency_s":0.267902,"redis_calls":7,"redis_duration_s":0.0024720000000000002,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001792,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00068,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024746,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.037917,"completed_at":"2026-07-04T12:39:26.035Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003537,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2199vm-test-run-nixbot-gitlab> gitlab # [  225.237217] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:26.038Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"71b635c636ba11713c0f7ab3","created_at":"2026-07-04T12:39:25.744Z","trace_propagation_headers":{"sentry-trace":"9c330060dd2f4ac98e5b541dc55c8d51-592ca9bb08e84302","baggage":"sentry-trace_id=9c330060dd2f4ac98e5b541dc55c8d51,sentry-release=Unknown"},"correlation_id":"01KWPJ839N05PX7EG89Y4MK6K4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:25.747Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-71b635c636ba11713c0f7ab3: start","job_status":"start","queue_duration_s":0.29053,"scheduling_latency_s":0.290586}2200vm-test-run-nixbot-gitlab> gitlab # [  225.263252] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:26.078Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"71b635c636ba11713c0f7ab3","created_at":"2026-07-04T12:39:25.744Z","trace_propagation_headers":{"sentry-trace":"9c330060dd2f4ac98e5b541dc55c8d51-592ca9bb08e84302","baggage":"sentry-trace_id=9c330060dd2f4ac98e5b541dc55c8d51,sentry-release=Unknown"},"correlation_id":"01KWPJ839N05PX7EG89Y4MK6K4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:25.747Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-71b635c636ba11713c0f7ab3: done: 0.040037 sec","job_status":"done","queue_duration_s":0.29053,"scheduling_latency_s":0.290586,"redis_calls":7,"redis_duration_s":0.00238,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001765,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000615,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026009,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.040037,"completed_at":"2026-07-04T12:39:26.078Z","load_balancing_strategy":"primary","db_duration_s":0.005395,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2201vm-test-run-nixbot-gitlab> gitlab # [  225.347130] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:26.079Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"eb1f9cb24b1c7970dea9e3a2","created_at":"2026-07-04T12:39:25.762Z","trace_propagation_headers":{"sentry-trace":"9c330060dd2f4ac98e5b541dc55c8d51-592ca9bb08e84302","baggage":"sentry-trace_id=9c330060dd2f4ac98e5b541dc55c8d51,sentry-release=Unknown"},"correlation_id":"01KWPJ839N05PX7EG89Y4MK6K4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:25.765Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-eb1f9cb24b1c7970dea9e3a2: start","job_status":"start","queue_duration_s":0.314033,"scheduling_latency_s":0.314057}2202vm-test-run-nixbot-gitlab> gitlab # [  225.354117] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:26.107Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"eb1f9cb24b1c7970dea9e3a2","created_at":"2026-07-04T12:39:25.762Z","trace_propagation_headers":{"sentry-trace":"9c330060dd2f4ac98e5b541dc55c8d51-592ca9bb08e84302","baggage":"sentry-trace_id=9c330060dd2f4ac98e5b541dc55c8d51,sentry-release=Unknown"},"correlation_id":"01KWPJ839N05PX7EG89Y4MK6K4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:25.765Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-eb1f9cb24b1c7970dea9e3a2: done: 0.027928 sec","job_status":"done","queue_duration_s":0.314033,"scheduling_latency_s":0.314057,"redis_calls":10,"redis_duration_s":0.003028,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001785,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000674,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000569,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018741,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027928,"completed_at":"2026-07-04T12:39:26.107Z","load_balancing_strategy":"primary","db_duration_s":0.003625,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2203vm-test-run-nixbot-gitlab> gitlab # [  225.491299] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:26.109Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"c621eebff8a4e8b4b46f1e49","created_at":"2026-07-04T12:39:25.766Z","trace_propagation_headers":{"sentry-trace":"9c330060dd2f4ac98e5b541dc55c8d51-592ca9bb08e84302","baggage":"sentry-trace_id=9c330060dd2f4ac98e5b541dc55c8d51,sentry-release=Unknown"},"correlation_id":"01KWPJ839N05PX7EG89Y4MK6K4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:25.770Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-c621eebff8a4e8b4b46f1e49: start","job_status":"start","queue_duration_s":0.339216,"scheduling_latency_s":0.339246}2204vm-test-run-nixbot-gitlab> gitlab # [  225.492835] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:26.129Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"c621eebff8a4e8b4b46f1e49","created_at":"2026-07-04T12:39:25.766Z","trace_propagation_headers":{"sentry-trace":"9c330060dd2f4ac98e5b541dc55c8d51-592ca9bb08e84302","baggage":"sentry-trace_id=9c330060dd2f4ac98e5b541dc55c8d51,sentry-release=Unknown"},"correlation_id":"01KWPJ839N05PX7EG89Y4MK6K4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:25.770Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-c621eebff8a4e8b4b46f1e49: done: 0.019891 sec","job_status":"done","queue_duration_s":0.339216,"scheduling_latency_s":0.339246,"redis_calls":7,"redis_duration_s":0.0016879999999999998,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001243,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000445,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011842,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019891,"completed_at":"2026-07-04T12:39:26.129Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003138,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2205vm-test-run-nixbot-gitlab> gitlab # [  225.519134] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:26.130Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"aa4dbbcaf7e3f1995b13ea06","created_at":"2026-07-04T12:39:25.771Z","trace_propagation_headers":{"sentry-trace":"9c330060dd2f4ac98e5b541dc55c8d51-592ca9bb08e84302","baggage":"sentry-trace_id=9c330060dd2f4ac98e5b541dc55c8d51,sentry-release=Unknown"},"correlation_id":"01KWPJ839N05PX7EG89Y4MK6K4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:25.775Z","job_size_bytes":19,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-aa4dbbcaf7e3f1995b13ea06: start","job_status":"start","queue_duration_s":0.355259,"scheduling_latency_s":0.355286}2206vm-test-run-nixbot-gitlab> gitlab # [  225.545339] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:26.150Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"aa4dbbcaf7e3f1995b13ea06","created_at":"2026-07-04T12:39:25.771Z","trace_propagation_headers":{"sentry-trace":"9c330060dd2f4ac98e5b541dc55c8d51-592ca9bb08e84302","baggage":"sentry-trace_id=9c330060dd2f4ac98e5b541dc55c8d51,sentry-release=Unknown"},"correlation_id":"01KWPJ839N05PX7EG89Y4MK6K4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:25.775Z","job_size_bytes":19,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-aa4dbbcaf7e3f1995b13ea06: done: 0.019835 sec","job_status":"done","queue_duration_s":0.355259,"scheduling_latency_s":0.355286,"redis_calls":8,"redis_duration_s":0.0018960000000000001,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001431,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000465,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012727,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019835,"completed_at":"2026-07-04T12:39:26.150Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00249,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2207vm-test-run-nixbot-gitlab> gitlab # [  225.614090] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:26.151Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"9462eb1999240bf56326fcd2","created_at":"2026-07-04T12:39:25.776Z","trace_propagation_headers":{"sentry-trace":"9c330060dd2f4ac98e5b541dc55c8d51-592ca9bb08e84302","baggage":"sentry-trace_id=9c330060dd2f4ac98e5b541dc55c8d51,sentry-release=Unknown"},"correlation_id":"01KWPJ839N05PX7EG89Y4MK6K4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:25.779Z","job_size_bytes":66,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-9462eb1999240bf56326fcd2: start","job_status":"start","queue_duration_s":0.371903,"scheduling_latency_s":0.371926}2208vm-test-run-nixbot-gitlab> gitlab # [  225.642054] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:26.171Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"9462eb1999240bf56326fcd2","created_at":"2026-07-04T12:39:25.776Z","trace_propagation_headers":{"sentry-trace":"9c330060dd2f4ac98e5b541dc55c8d51-592ca9bb08e84302","baggage":"sentry-trace_id=9c330060dd2f4ac98e5b541dc55c8d51,sentry-release=Unknown"},"correlation_id":"01KWPJ839N05PX7EG89Y4MK6K4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:25.779Z","job_size_bytes":66,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-9462eb1999240bf56326fcd2: done: 0.019822 sec","job_status":"done","queue_duration_s":0.371903,"scheduling_latency_s":0.371926,"redis_calls":7,"redis_duration_s":0.0017559999999999997,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001267,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000489,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012666,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019822,"completed_at":"2026-07-04T12:39:26.171Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002697,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2209vm-test-run-nixbot-gitlab> gitlab # [  225.712181] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:26.173Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"b99788f6c69019d163be2ae4","created_at":"2026-07-04T12:39:25.781Z","trace_propagation_headers":{"sentry-trace":"9c330060dd2f4ac98e5b541dc55c8d51-592ca9bb08e84302","baggage":"sentry-trace_id=9c330060dd2f4ac98e5b541dc55c8d51,sentry-release=Unknown"},"correlation_id":"01KWPJ839N05PX7EG89Y4MK6K4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:25.784Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-b99788f6c69019d163be2ae4: start","job_status":"start","queue_duration_s":0.388651,"scheduling_latency_s":0.388677}2210vm-test-run-nixbot-gitlab> gitlab # [  225.783157] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:26.198Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"b99788f6c69019d163be2ae4","created_at":"2026-07-04T12:39:25.781Z","trace_propagation_headers":{"sentry-trace":"9c330060dd2f4ac98e5b541dc55c8d51-592ca9bb08e84302","baggage":"sentry-trace_id=9c330060dd2f4ac98e5b541dc55c8d51,sentry-release=Unknown"},"correlation_id":"01KWPJ839N05PX7EG89Y4MK6K4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:25.784Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-b99788f6c69019d163be2ae4: done: 0.025196 sec","job_status":"done","queue_duration_s":0.388651,"scheduling_latency_s":0.388677,"redis_calls":7,"redis_duration_s":0.001552,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001126,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000426,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017671,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025196,"completed_at":"2026-07-04T12:39:26.198Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00324,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2211vm-test-run-nixbot-gitlab> gitlab # [  225.804810] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:26.199Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"2e43482aa481e86a7ee3473c","created_at":"2026-07-04T12:39:25.865Z","trace_propagation_headers":{"sentry-trace":"9c330060dd2f4ac98e5b541dc55c8d51-592ca9bb08e84302","baggage":"sentry-trace_id=9c330060dd2f4ac98e5b541dc55c8d51,sentry-release=Unknown"},"correlation_id":"01KWPJ839N05PX7EG89Y4MK6K4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:25.867Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-2e43482aa481e86a7ee3473c: start","job_status":"start","queue_duration_s":0.331723,"scheduling_latency_s":0.331746}2212vm-test-run-nixbot-gitlab> gitlab # [  225.831185] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:26.218Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"2e43482aa481e86a7ee3473c","created_at":"2026-07-04T12:39:25.865Z","trace_propagation_headers":{"sentry-trace":"9c330060dd2f4ac98e5b541dc55c8d51-592ca9bb08e84302","baggage":"sentry-trace_id=9c330060dd2f4ac98e5b541dc55c8d51,sentry-release=Unknown"},"correlation_id":"01KWPJ839N05PX7EG89Y4MK6K4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:25.867Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-2e43482aa481e86a7ee3473c: done: 0.018417 sec","job_status":"done","queue_duration_s":0.331723,"scheduling_latency_s":0.331746,"redis_calls":7,"redis_duration_s":0.0016259999999999998,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001156,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00047,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011226,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018417,"completed_at":"2026-07-04T12:39:26.217Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003141,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2213vm-test-run-nixbot-gitlab> gitlab # [  225.916141] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:26.219Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"a8ca65d7e8c7ada6790f3790","created_at":"2026-07-04T12:39:25.868Z","trace_propagation_headers":{"sentry-trace":"9c330060dd2f4ac98e5b541dc55c8d51-592ca9bb08e84302","baggage":"sentry-trace_id=9c330060dd2f4ac98e5b541dc55c8d51,sentry-release=Unknown"},"correlation_id":"01KWPJ839N05PX7EG89Y4MK6K4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:25.871Z","job_size_bytes":6,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-a8ca65d7e8c7ada6790f3790: start","job_status":"start","queue_duration_s":0.34806,"scheduling_latency_s":0.34809}2214vm-test-run-nixbot-gitlab> gitlab # [  225.923059] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:26.231Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"a8ca65d7e8c7ada6790f3790","created_at":"2026-07-04T12:39:25.868Z","trace_propagation_headers":{"sentry-trace":"9c330060dd2f4ac98e5b541dc55c8d51-592ca9bb08e84302","baggage":"sentry-trace_id=9c330060dd2f4ac98e5b541dc55c8d51,sentry-release=Unknown"},"correlation_id":"01KWPJ839N05PX7EG89Y4MK6K4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:25.871Z","job_size_bytes":6,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-a8ca65d7e8c7ada6790f3790: done: 0.012563 sec","job_status":"done","queue_duration_s":0.34806,"scheduling_latency_s":0.34809,"redis_calls":7,"redis_duration_s":0.001572,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001154,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000418,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008077,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012563,"completed_at":"2026-07-04T12:39:26.231Z","load_balancing_strategy":"primary","db_duration_s":0.000375,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2215vm-test-run-nixbot-gitlab> gitlab # [  225.991310] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:26.233Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"0510053d41c25243b36eed7f","created_at":"2026-07-04T12:39:25.872Z","trace_propagation_headers":{"sentry-trace":"9c330060dd2f4ac98e5b541dc55c8d51-592ca9bb08e84302","baggage":"sentry-trace_id=9c330060dd2f4ac98e5b541dc55c8d51,sentry-release=Unknown"},"correlation_id":"01KWPJ839N05PX7EG89Y4MK6K4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:25.879Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-0510053d41c25243b36eed7f: start","job_status":"start","queue_duration_s":0.353867,"scheduling_latency_s":0.353887}2216vm-test-run-nixbot-gitlab> gitlab # [  226.062086] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:26.258Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"0510053d41c25243b36eed7f","created_at":"2026-07-04T12:39:25.872Z","trace_propagation_headers":{"sentry-trace":"9c330060dd2f4ac98e5b541dc55c8d51-592ca9bb08e84302","baggage":"sentry-trace_id=9c330060dd2f4ac98e5b541dc55c8d51,sentry-release=Unknown"},"correlation_id":"01KWPJ839N05PX7EG89Y4MK6K4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:25.879Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-0510053d41c25243b36eed7f: done: 0.025103 sec","job_status":"done","queue_duration_s":0.353867,"scheduling_latency_s":0.353887,"redis_calls":10,"redis_duration_s":0.002212,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000444,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001329,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000439,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015541,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000346133,"duration_s":0.025103,"completed_at":"2026-07-04T12:39:26.258Z","load_balancing_strategy":"primary","db_duration_s":0.004026,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2217vm-test-run-nixbot-gitlab> gitlab # [  226.082380] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:26.259Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"05b5e7b7cf10560c84b49d4f","created_at":"2026-07-04T12:39:25.922Z","trace_propagation_headers":{"sentry-trace":"9c330060dd2f4ac98e5b541dc55c8d51-592ca9bb08e84302","baggage":"sentry-trace_id=9c330060dd2f4ac98e5b541dc55c8d51,sentry-release=Unknown"},"correlation_id":"01KWPJ839N05PX7EG89Y4MK6K4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:25.923Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-05b5e7b7cf10560c84b49d4f: start","job_status":"start","queue_duration_s":0.335749,"scheduling_latency_s":0.33577}2218vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/98211a6541adb8d86116a7680ad181c0d8549f29/statuses, in 1.13 seconds)2219vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': '98211a6541adb8d86116a7680ad181c0d8549f29', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-04T12:39:17.921Z', 'started_at': None, 'finished_at': '2026-07-04T12:39:25.364Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '98211a6541adb8d86116a7680ad181c0d8549f29', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-07-04T12:39:25.557Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2220vm-test-run-nixbot-gitlab> gitlab # [  226.108122] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:26.370Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"05b5e7b7cf10560c84b49d4f","created_at":"2026-07-04T12:39:25.922Z","trace_propagation_headers":{"sentry-trace":"9c330060dd2f4ac98e5b541dc55c8d51-592ca9bb08e84302","baggage":"sentry-trace_id=9c330060dd2f4ac98e5b541dc55c8d51,sentry-release=Unknown"},"correlation_id":"01KWPJ839N05PX7EG89Y4MK6K4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:25.923Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-05b5e7b7cf10560c84b49d4f: done: 0.111382 sec","job_status":"done","queue_duration_s":0.335749,"scheduling_latency_s":0.33577,"gitaly_calls":3,"gitaly_duration_s":0.009089,"redis_calls":44,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.007633,"redis_read_bytes":98,"redis_write_bytes":6013,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000979,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.001945,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":2,"redis_queues_duration_s":0.000275,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1132,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001604,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000239,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000491,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001121,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":20,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.077331,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.088616291,"duration_s":0.111382,"completed_at":"2026-07-04T12:39:26.370Z","load_balancing_strategy":"primary","db_duration_s":0.024248,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2221vm-test-run-nixbot-gitlab> gitlab # [  226.191077] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:26.372Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"6198bbd1aabb87a6fc349fd8","created_at":"2026-07-04T12:39:26.303Z","trace_propagation_headers":{"sentry-trace":"9c330060dd2f4ac98e5b541dc55c8d51-0e94c58a4ba049f1","baggage":"sentry-trace_id=9c330060dd2f4ac98e5b541dc55c8d51,sentry-release=Unknown"},"correlation_id":"01KWPJ839N05PX7EG89Y4MK6K4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:26.304Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-6198bbd1aabb87a6fc349fd8: start","job_status":"start","queue_duration_s":0.067679,"scheduling_latency_s":0.067701}2222vm-test-run-nixbot-gitlab> gitlab # [  226.273525] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:26.397Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"6198bbd1aabb87a6fc349fd8","created_at":"2026-07-04T12:39:26.303Z","trace_propagation_headers":{"sentry-trace":"9c330060dd2f4ac98e5b541dc55c8d51-0e94c58a4ba049f1","baggage":"sentry-trace_id=9c330060dd2f4ac98e5b541dc55c8d51,sentry-release=Unknown"},"correlation_id":"01KWPJ839N05PX7EG89Y4MK6K4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:26.304Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-6198bbd1aabb87a6fc349fd8: done: 0.024941 sec","job_status":"done","queue_duration_s":0.067679,"scheduling_latency_s":0.067701,"redis_calls":7,"redis_duration_s":0.0016099999999999999,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001127,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000483,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017303,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024941,"completed_at":"2026-07-04T12:39:26.397Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003041,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2223vm-test-run-nixbot-gitlab> gitlab # [  226.283375] gitaly[1392]: time="2026-07-04T12:39:26.314Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ839N05PX7EG89Y4MK6K4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-04T12:39:56.312" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:26.313" grpc.time_ms=0.902 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2224vm-test-run-nixbot-gitlab> gitlab # [  226.361654] gitlab-workhorse[1293]: gitlab 127.0.0.1 - - [2026/07/04:12:39:27 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/98211a6541adb8d86116a7680ad181c0d8549f29/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 792225vm-test-run-nixbot-gitlab> gitlab # [  226.363089] gitaly[1392]: time="2026-07-04T12:39:26.335Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ839N05PX7EG89Y4MK6K4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T12:39:56.334" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:26.335" grpc.time_ms=0.526 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2226vm-test-run-nixbot-gitlab> gitlab # [  226.363980] gitaly[1392]: time="2026-07-04T12:39:26.357Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ839N05PX7EG89Y4MK6K4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T12:39:56.355" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:26.356" grpc.time_ms=0.592 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2227vm-test-run-nixbot-gitlab> gitlab # [  226.365113] gitaly[1392]: time="2026-07-04T12:39:27.169Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ85427AY8967DPB9V80N7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T12:39:57.068" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:27.168" grpc.time_ms=0.697 pid=1392 remote_ip=127.0.0.1 span.kind=server system=grpc username=root2228vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/98211a6541adb8d86116a7680ad181c0d8549f29/statuses2229vm-test-run-nixbot-gitlab> gitlab # [  227.369800] gitaly[1392]: time="2026-07-04T12:39:28.320Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ867PZ6VWYXA2FSRWN3AQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T12:39:58.217" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:28.318" grpc.time_ms=1.521 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2230vm-test-run-nixbot-gitlab> gitlab # [  227.393727] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:28 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/98211a6541adb8d86116a7680ad181c0d8549f29/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 972231vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/98211a6541adb8d86116a7680ad181c0d8549f29/statuses, in 0.18 seconds)2232vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': '98211a6541adb8d86116a7680ad181c0d8549f29', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-04T12:39:17.921Z', 'started_at': None, 'finished_at': '2026-07-04T12:39:25.364Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '98211a6541adb8d86116a7680ad181c0d8549f29', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-07-04T12:39:25.557Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2233vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/98211a6541adb8d86116a7680ad181c0d8549f29/statuses2234vm-test-run-nixbot-gitlab> gitlab # [  228.537992] gitaly[1392]: time="2026-07-04T12:39:29.488Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ87CFXJNH2KN7J7A5PBTT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T12:39:59.086" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:29.487" grpc.time_ms=1.202 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2235vm-test-run-nixbot-gitlab> gitlab # [  228.559909] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:29 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/98211a6541adb8d86116a7680ad181c0d8549f29/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 872236vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/98211a6541adb8d86116a7680ad181c0d8549f29/statuses, in 0.17 seconds)2237vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': '98211a6541adb8d86116a7680ad181c0d8549f29', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-04T12:39:17.921Z', 'started_at': None, 'finished_at': '2026-07-04T12:39:25.364Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '98211a6541adb8d86116a7680ad181c0d8549f29', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-07-04T12:39:25.557Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2238vm-test-run-nixbot-gitlab> gitlab # [  228.909988] nix-daemon[3304]: accepted connection from pid 3654, user nixbot2239vm-test-run-nixbot-gitlab> gitlab # [  229.000983] gitaly[1392]: time="2026-07-04T12:39:29.952Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ87VD2ZGYBHN576S02EVH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T12:39:59.750" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:29.951" grpc.time_ms=0.94 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2240vm-test-run-nixbot-gitlab> gitlab # [  229.041126] gitaly[1392]: time="2026-07-04T12:39:29.992Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ87VD2ZGYBHN576S02EVH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-04T12:39:39.254" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T12:39:29.984" grpc.time_ms=7.738 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2241vm-test-run-nixbot-gitlab> gitlab # [  229.136430] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:30.085Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"702575660d7bcc209dcb9a90","created_at":"2026-07-04T12:39:30.075Z","trace_propagation_headers":{"sentry-trace":"5d70842081f045d3b8986a63683c0740-324f95f8fa004a55","baggage":"sentry-trace_id=5d70842081f045d3b8986a63683c0740,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPJ87VD2ZGYBHN576S02EVH","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:30.082Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-702575660d7bcc209dcb9a90: start","job_status":"start","queue_duration_s":0.003098,"scheduling_latency_s":0.003189}2242vm-test-run-nixbot-gitlab> gitlab # [  229.209112] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:30 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/98211a6541adb8d86116a7680ad181c0d8549f29 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 2572243vm-test-run-nixbot-gitlab> gitlab # [  229.220339] nixbot[3144]: {"ts": "2026-07-04T12:39:30.161Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/98211a6541adb8d86116a7680ad181c0d8549f29 \"HTTP/1.1 201 Created\""}2244vm-test-run-nixbot-gitlab> gitlab # [  229.374816] gitaly[1392]: time="2026-07-04T12:39:30.325Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPJ87VD2ZGYBHN576S02EVH grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T12:39:30.324" grpc.time_ms=0.925 pid=1392 span.kind=server system=grpc2245vm-test-run-nixbot-gitlab> gitlab # [  229.434825] gitaly[1392]: time="2026-07-04T12:39:30.386Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPJ87VD2ZGYBHN576S02EVH grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T12:39:30.385" grpc.time_ms=0.406 pid=1392 span.kind=server system=grpc2246vm-test-run-nixbot-gitlab> gitlab # [  229.463738] nix-daemon[3304]: accepted connection from pid 3681, user nixbot2247vm-test-run-nixbot-gitlab> gitlab # [  229.480460] gitaly[1392]: time="2026-07-04T12:39:30.432Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPJ87VD2ZGYBHN576S02EVH grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T12:39:30.431" grpc.time_ms=0.339 pid=1392 span.kind=server system=grpc2248vm-test-run-nixbot-gitlab> gitlab # [  229.498858] gitaly[1392]: time="2026-07-04T12:39:30.450Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ87VD2ZGYBHN576S02EVH forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-04T12:40:00.268" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-04T12:39:30.269" operations="[repack_refs]" pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2249vm-test-run-nixbot-gitlab> gitlab # [  229.509237] gitaly[1392]: time="2026-07-04T12:39:30.460Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ87VD2ZGYBHN576S02EVH diskcache=e2318991-99bb-4dcd-b45c-0eccbeeeb20b grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-04T12:40:00.268" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-04T12:39:30.269" pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2250vm-test-run-nixbot-gitlab> gitlab # [  229.510467] gitaly[1392]: time="2026-07-04T12:39:30.461Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=150 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7166 command.oublock=0 command.real_time_ms=178 command.spawn_token_fork_ms=1 command.system_time_ms=116 command.user_time_ms=34 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ87VD2ZGYBHN576S02EVH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-04T12:40:00.268" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T12:39:30.269" grpc.time_ms=191.441 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2251vm-test-run-nixbot-gitlab> gitlab # [  229.555535] gitaly[1392]: time="2026-07-04T12:39:30.505Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ87VD2ZGYBHN576S02EVH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-04T12:40:00.268" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-04T12:39:30.269" pid=1392 remote_ip="2001:db8:1::1" repository_info="{false {6 704 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-07-04 12:39:01.901262987 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root2252vm-test-run-nixbot-gitlab> gitlab # [  229.577854] gitaly[1392]: time="2026-07-04T12:39:30.529Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ87VD2ZGYBHN576S02EVH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-04T12:40:00.028" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:30.528" grpc.time_ms=0.701 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2253vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/98211a6541adb8d86116a7680ad181c0d8549f29/statuses2254vm-test-run-nixbot-gitlab> gitlab # [  229.590806] gitaly[1392]: time="2026-07-04T12:39:30.542Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ87VD2ZGYBHN576S02EVH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-04T12:40:00.268" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-04T12:39:30.269" pid=1392 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root2255vm-test-run-nixbot-gitlab> gitlab # [  229.592927] gitaly[1392]: time="2026-07-04T12:39:30.544Z" level=info msg="optimized repository" clean_stale_data_time=36.23142ms clean_work_trees_time="250.032µs" commit_graph_time="8.101µs" correlation_id=01KWPJ87VD2ZGYBHN576S02EVH gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="8.381µs" pid=1392 prune_time="7.822µs" relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="8.381µs" storage_name=default system=housekeeping total_time=38.077465ms2256vm-test-run-nixbot-gitlab> gitlab # [  229.594262] gitaly[1392]: time="2026-07-04T12:39:30.544Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ87VD2ZGYBHN576S02EVH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-04T12:40:00.268" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-04T12:39:30.269" pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2257vm-test-run-nixbot-gitlab> gitlab # [  229.636533] gitaly[1392]: time="2026-07-04T12:39:30.588Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ87VD2ZGYBHN576S02EVH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T12:40:00.086" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:30.587" grpc.time_ms=0.699 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2258vm-test-run-nixbot-gitlab> gitlab # [  229.679826] gitaly[1392]: time="2026-07-04T12:39:30.631Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPJ88DEBGKPHBQ6E5CND4HR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T12:39:30.620" grpc.time_ms=10.801 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc2259vm-test-run-nixbot-gitlab> gitlab # [  229.685265] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:30 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 1532260vm-test-run-nixbot-gitlab> gitlab # [  229.708827] gitaly[1392]: time="2026-07-04T12:39:30.658Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ87VD2ZGYBHN576S02EVH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T12:40:00.156" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:30.657" grpc.time_ms=0.835 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2261vm-test-run-nixbot-gitlab> gitlab # [  229.740098] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:30.691Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"702575660d7bcc209dcb9a90","created_at":"2026-07-04T12:39:30.075Z","trace_propagation_headers":{"sentry-trace":"5d70842081f045d3b8986a63683c0740-324f95f8fa004a55","baggage":"sentry-trace_id=5d70842081f045d3b8986a63683c0740,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPJ87VD2ZGYBHN576S02EVH","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:30.082Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-702575660d7bcc209dcb9a90: done: 0.605946 sec","job_status":"done","queue_duration_s":0.003098,"scheduling_latency_s":0.003189,"gitaly_calls":4,"gitaly_duration_s":0.274673,"redis_calls":83,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.027225,"redis_read_bytes":173,"redis_write_bytes":20928,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.002799,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":21,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.004404,"redis_cache_read_bytes":39,"redis_cache_write_bytes":1840,"redis_queues_calls":23,"redis_queues_duration_s":0.004889,"redis_queues_read_bytes":25,"redis_queues_write_bytes":13381,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.007321,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3094,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000547,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00063,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.003836,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":36,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.079,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":21,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.013,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.017,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.033,"db_ci_duration_s":0.046,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.168264,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.493365218,"duration_s":0.605946,"completed_at":"2026-07-04T12:39:30.691Z","load_balancing_strategy":"primary","db_duration_s":0.084053,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2262vm-test-run-nixbot-gitlab> gitlab # [  229.751878] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:30.697Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"b8ae2aeb023df8ab1312c287","created_at":"2026-07-04T12:39:30.252Z","trace_propagation_headers":{"sentry-trace":"5d70842081f045d3b8986a63683c0740-87376907679c46a2","baggage":"sentry-trace_id=5d70842081f045d3b8986a63683c0740,sentry-release=Unknown"},"correlation_id":"01KWPJ87VD2ZGYBHN576S02EVH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:30.257Z","job_size_bytes":23,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-b8ae2aeb023df8ab1312c287: start","job_status":"start","queue_duration_s":0.439185,"scheduling_latency_s":0.439212}2263vm-test-run-nixbot-gitlab> gitlab # [  229.778770] gitaly[1392]: time="2026-07-04T12:39:30.725Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ88J9KTYEHPZQMJXH6ERR grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T12:39:30.704" pid=1392 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc2264vm-test-run-nixbot-gitlab> gitlab # [  229.794546] gitlab-workhorse[1293]: time="2026-07-04T12:39:30Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672265vm-test-run-nixbot-gitlab> gitlab # [  229.797349] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:30 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 942266vm-test-run-nixbot-gitlab> gitlab # [  229.800260] gitaly[1392]: time="2026-07-04T12:39:30.726Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ88J9KTYEHPZQMJXH6ERR grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T12:39:30.704" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc2267vm-test-run-nixbot-gitlab> gitlab # [  229.822359] gitaly[1392]: time="2026-07-04T12:39:30.727Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ88J9KTYEHPZQMJXH6ERR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T12:39:30.704" grpc.time_ms=21.754 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc2268vm-test-run-nixbot-gitlab> gitlab # [  229.845511] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:30.753Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"b8ae2aeb023df8ab1312c287","created_at":"2026-07-04T12:39:30.252Z","trace_propagation_headers":{"sentry-trace":"5d70842081f045d3b8986a63683c0740-87376907679c46a2","baggage":"sentry-trace_id=5d70842081f045d3b8986a63683c0740,sentry-release=Unknown"},"correlation_id":"01KWPJ87VD2ZGYBHN576S02EVH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:30.257Z","job_size_bytes":23,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-b8ae2aeb023df8ab1312c287: done: 0.056655 sec","job_status":"done","queue_duration_s":0.439185,"scheduling_latency_s":0.439212,"redis_calls":7,"redis_duration_s":0.008019,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003188,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004831,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027568,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.056655,"completed_at":"2026-07-04T12:39:30.753Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014648,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2269vm-test-run-nixbot-gitlab> gitlab # [  229.936265] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:30.755Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"e35fd0e06f70a807b710ea14","created_at":"2026-07-04T12:39:30.469Z","trace_propagation_headers":{"sentry-trace":"5d70842081f045d3b8986a63683c0740-87376907679c46a2","baggage":"sentry-trace_id=5d70842081f045d3b8986a63683c0740,sentry-release=Unknown"},"correlation_id":"01KWPJ87VD2ZGYBHN576S02EVH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:30.471Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-e35fd0e06f70a807b710ea14: start","job_status":"start","queue_duration_s":0.283996,"scheduling_latency_s":0.284026}2270vm-test-run-nixbot-gitlab> gitlab # [  229.939285] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:30.776Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"e35fd0e06f70a807b710ea14","created_at":"2026-07-04T12:39:30.469Z","trace_propagation_headers":{"sentry-trace":"5d70842081f045d3b8986a63683c0740-87376907679c46a2","baggage":"sentry-trace_id=5d70842081f045d3b8986a63683c0740,sentry-release=Unknown"},"correlation_id":"01KWPJ87VD2ZGYBHN576S02EVH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:30.471Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-e35fd0e06f70a807b710ea14: done: 0.021331 sec","job_status":"done","queue_duration_s":0.283996,"scheduling_latency_s":0.284026,"redis_calls":7,"redis_duration_s":0.001678,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001222,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000456,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012822,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021331,"completed_at":"2026-07-04T12:39:30.776Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003508,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2271vm-test-run-nixbot-gitlab> gitlab # [  230.009188] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:30.778Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"e48163e9d59c5ede2d4d75ce","created_at":"2026-07-04T12:39:30.479Z","trace_propagation_headers":{"sentry-trace":"5d70842081f045d3b8986a63683c0740-87376907679c46a2","baggage":"sentry-trace_id=5d70842081f045d3b8986a63683c0740,sentry-release=Unknown"},"correlation_id":"01KWPJ87VD2ZGYBHN576S02EVH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:30.481Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-e48163e9d59c5ede2d4d75ce: start","job_status":"start","queue_duration_s":0.296739,"scheduling_latency_s":0.296763}2272vm-test-run-nixbot-gitlab> gitlab # [  230.101154] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:30.851Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"e48163e9d59c5ede2d4d75ce","created_at":"2026-07-04T12:39:30.479Z","trace_propagation_headers":{"sentry-trace":"5d70842081f045d3b8986a63683c0740-87376907679c46a2","baggage":"sentry-trace_id=5d70842081f045d3b8986a63683c0740,sentry-release=Unknown"},"correlation_id":"01KWPJ87VD2ZGYBHN576S02EVH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:30.481Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-e48163e9d59c5ede2d4d75ce: done: 0.072466 sec","job_status":"done","queue_duration_s":0.296739,"scheduling_latency_s":0.296763,"redis_calls":7,"redis_duration_s":0.002175,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001558,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000617,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.049,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.049,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018435,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.072466,"completed_at":"2026-07-04T12:39:30.851Z","load_balancing_strategy":"primary","db_duration_s":0.048939,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2273vm-test-run-nixbot-gitlab> gitlab # [  230.103146] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:30.852Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"fb436b36ea0367f3a3ab016e","created_at":"2026-07-04T12:39:30.488Z","trace_propagation_headers":{"sentry-trace":"5d70842081f045d3b8986a63683c0740-87376907679c46a2","baggage":"sentry-trace_id=5d70842081f045d3b8986a63683c0740,sentry-release=Unknown"},"correlation_id":"01KWPJ87VD2ZGYBHN576S02EVH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:30.490Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-fb436b36ea0367f3a3ab016e: start","job_status":"start","queue_duration_s":0.362374,"scheduling_latency_s":0.362401}2274vm-test-run-nixbot-gitlab> gitlab # [  230.125132] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:30.877Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"fb436b36ea0367f3a3ab016e","created_at":"2026-07-04T12:39:30.488Z","trace_propagation_headers":{"sentry-trace":"5d70842081f045d3b8986a63683c0740-87376907679c46a2","baggage":"sentry-trace_id=5d70842081f045d3b8986a63683c0740,sentry-release=Unknown"},"correlation_id":"01KWPJ87VD2ZGYBHN576S02EVH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:30.490Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-fb436b36ea0367f3a3ab016e: done: 0.024764 sec","job_status":"done","queue_duration_s":0.362374,"scheduling_latency_s":0.362401,"redis_calls":10,"redis_duration_s":0.002284,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001154,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000641,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000489,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015681,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024764,"completed_at":"2026-07-04T12:39:30.877Z","load_balancing_strategy":"primary","db_duration_s":0.003074,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2275vm-test-run-nixbot-gitlab> gitlab # [  230.245576] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:30.878Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"b225f9ad18677e41b200b60a","created_at":"2026-07-04T12:39:30.490Z","trace_propagation_headers":{"sentry-trace":"5d70842081f045d3b8986a63683c0740-87376907679c46a2","baggage":"sentry-trace_id=5d70842081f045d3b8986a63683c0740,sentry-release=Unknown"},"correlation_id":"01KWPJ87VD2ZGYBHN576S02EVH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:30.491Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-b225f9ad18677e41b200b60a: start","job_status":"start","queue_duration_s":0.386728,"scheduling_latency_s":0.386751}2276vm-test-run-nixbot-gitlab> gitlab # [  230.246749] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:30.899Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"b225f9ad18677e41b200b60a","created_at":"2026-07-04T12:39:30.490Z","trace_propagation_headers":{"sentry-trace":"5d70842081f045d3b8986a63683c0740-87376907679c46a2","baggage":"sentry-trace_id=5d70842081f045d3b8986a63683c0740,sentry-release=Unknown"},"correlation_id":"01KWPJ87VD2ZGYBHN576S02EVH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:30.491Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-b225f9ad18677e41b200b60a: done: 0.020389 sec","job_status":"done","queue_duration_s":0.386728,"scheduling_latency_s":0.386751,"redis_calls":7,"redis_duration_s":0.0020480000000000003,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00161,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000438,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011603,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020389,"completed_at":"2026-07-04T12:39:30.899Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003619,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2277vm-test-run-nixbot-gitlab> gitlab # [  230.335893] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:30.900Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"48e7ea01a5ee2ef906a750ab","created_at":"2026-07-04T12:39:30.492Z","trace_propagation_headers":{"sentry-trace":"5d70842081f045d3b8986a63683c0740-87376907679c46a2","baggage":"sentry-trace_id=5d70842081f045d3b8986a63683c0740,sentry-release=Unknown"},"correlation_id":"01KWPJ87VD2ZGYBHN576S02EVH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:30.494Z","job_size_bytes":19,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-48e7ea01a5ee2ef906a750ab: start","job_status":"start","queue_duration_s":0.405567,"scheduling_latency_s":0.405589}2278vm-test-run-nixbot-gitlab> gitlab # [  230.337165] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:30.924Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"48e7ea01a5ee2ef906a750ab","created_at":"2026-07-04T12:39:30.492Z","trace_propagation_headers":{"sentry-trace":"5d70842081f045d3b8986a63683c0740-87376907679c46a2","baggage":"sentry-trace_id=5d70842081f045d3b8986a63683c0740,sentry-release=Unknown"},"correlation_id":"01KWPJ87VD2ZGYBHN576S02EVH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:30.494Z","job_size_bytes":19,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-48e7ea01a5ee2ef906a750ab: done: 0.023824 sec","job_status":"done","queue_duration_s":0.405567,"scheduling_latency_s":0.405589,"redis_calls":8,"redis_duration_s":0.0021,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00167,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00043,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013668,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023824,"completed_at":"2026-07-04T12:39:30.924Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005651,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2279vm-test-run-nixbot-gitlab> gitlab # [  230.381259] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:30.925Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"6e20c0dc6d4816b5fd5d924d","created_at":"2026-07-04T12:39:30.495Z","trace_propagation_headers":{"sentry-trace":"5d70842081f045d3b8986a63683c0740-87376907679c46a2","baggage":"sentry-trace_id=5d70842081f045d3b8986a63683c0740,sentry-release=Unknown"},"correlation_id":"01KWPJ87VD2ZGYBHN576S02EVH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:30.497Z","job_size_bytes":66,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-6e20c0dc6d4816b5fd5d924d: start","job_status":"start","queue_duration_s":0.428649,"scheduling_latency_s":0.428672}2280vm-test-run-nixbot-gitlab> gitlab # [  230.451064] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:30.943Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"6e20c0dc6d4816b5fd5d924d","created_at":"2026-07-04T12:39:30.495Z","trace_propagation_headers":{"sentry-trace":"5d70842081f045d3b8986a63683c0740-87376907679c46a2","baggage":"sentry-trace_id=5d70842081f045d3b8986a63683c0740,sentry-release=Unknown"},"correlation_id":"01KWPJ87VD2ZGYBHN576S02EVH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:30.497Z","job_size_bytes":66,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-6e20c0dc6d4816b5fd5d924d: done: 0.018298 sec","job_status":"done","queue_duration_s":0.428649,"scheduling_latency_s":0.428672,"redis_calls":7,"redis_duration_s":0.0018019999999999998,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001285,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000517,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011961,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018298,"completed_at":"2026-07-04T12:39:30.943Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001097,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2281vm-test-run-nixbot-gitlab> gitlab # [  230.532303] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:30.945Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"ae9c1de6713aab44a82896c7","created_at":"2026-07-04T12:39:30.497Z","trace_propagation_headers":{"sentry-trace":"5d70842081f045d3b8986a63683c0740-87376907679c46a2","baggage":"sentry-trace_id=5d70842081f045d3b8986a63683c0740,sentry-release=Unknown"},"correlation_id":"01KWPJ87VD2ZGYBHN576S02EVH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:30.499Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-ae9c1de6713aab44a82896c7: start","job_status":"start","queue_duration_s":0.446601,"scheduling_latency_s":0.446666}2282vm-test-run-nixbot-gitlab> gitlab # [  230.533829] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:30.972Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"ae9c1de6713aab44a82896c7","created_at":"2026-07-04T12:39:30.497Z","trace_propagation_headers":{"sentry-trace":"5d70842081f045d3b8986a63683c0740-87376907679c46a2","baggage":"sentry-trace_id=5d70842081f045d3b8986a63683c0740,sentry-release=Unknown"},"correlation_id":"01KWPJ87VD2ZGYBHN576S02EVH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:30.499Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-ae9c1de6713aab44a82896c7: done: 0.026821 sec","job_status":"done","queue_duration_s":0.446601,"scheduling_latency_s":0.446666,"redis_calls":7,"redis_duration_s":0.0018,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001219,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000581,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017269,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026821,"completed_at":"2026-07-04T12:39:30.972Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00467,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2283vm-test-run-nixbot-gitlab> gitlab # [  230.612193] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:30.974Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"efd051e197760f9a6c28a67c","created_at":"2026-07-04T12:39:30.593Z","trace_propagation_headers":{"sentry-trace":"5d70842081f045d3b8986a63683c0740-87376907679c46a2","baggage":"sentry-trace_id=5d70842081f045d3b8986a63683c0740,sentry-release=Unknown"},"correlation_id":"01KWPJ87VD2ZGYBHN576S02EVH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:30.594Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-efd051e197760f9a6c28a67c: start","job_status":"start","queue_duration_s":0.379099,"scheduling_latency_s":0.379123}2284vm-test-run-nixbot-gitlab> gitlab # [  230.613714] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:30.997Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"efd051e197760f9a6c28a67c","created_at":"2026-07-04T12:39:30.593Z","trace_propagation_headers":{"sentry-trace":"5d70842081f045d3b8986a63683c0740-87376907679c46a2","baggage":"sentry-trace_id=5d70842081f045d3b8986a63683c0740,sentry-release=Unknown"},"correlation_id":"01KWPJ87VD2ZGYBHN576S02EVH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:30.594Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-efd051e197760f9a6c28a67c: done: 0.023276 sec","job_status":"done","queue_duration_s":0.379099,"scheduling_latency_s":0.379123,"redis_calls":7,"redis_duration_s":0.002079,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001561,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000518,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011586,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023276,"completed_at":"2026-07-04T12:39:30.997Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003317,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2285vm-test-run-nixbot-gitlab> gitlab # [  230.685230] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:30.998Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"21b213cbf263143496e93fc9","created_at":"2026-07-04T12:39:30.595Z","trace_propagation_headers":{"sentry-trace":"5d70842081f045d3b8986a63683c0740-87376907679c46a2","baggage":"sentry-trace_id=5d70842081f045d3b8986a63683c0740,sentry-release=Unknown"},"correlation_id":"01KWPJ87VD2ZGYBHN576S02EVH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:30.596Z","job_size_bytes":6,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-21b213cbf263143496e93fc9: start","job_status":"start","queue_duration_s":0.401741,"scheduling_latency_s":0.401763}2286vm-test-run-nixbot-gitlab> gitlab # [  230.686804] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:31.011Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"21b213cbf263143496e93fc9","created_at":"2026-07-04T12:39:30.595Z","trace_propagation_headers":{"sentry-trace":"5d70842081f045d3b8986a63683c0740-87376907679c46a2","baggage":"sentry-trace_id=5d70842081f045d3b8986a63683c0740,sentry-release=Unknown"},"correlation_id":"01KWPJ87VD2ZGYBHN576S02EVH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:30.596Z","job_size_bytes":6,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-21b213cbf263143496e93fc9: done: 0.013309 sec","job_status":"done","queue_duration_s":0.401741,"scheduling_latency_s":0.401763,"redis_calls":7,"redis_duration_s":0.001733,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001202,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000531,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007895,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013309,"completed_at":"2026-07-04T12:39:31.011Z","load_balancing_strategy":"primary","db_duration_s":0.000347,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2287vm-test-run-nixbot-gitlab> gitlab # [  230.816048] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:31.013Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"43fdf99ec1c4d24b78f2c3eb","created_at":"2026-07-04T12:39:30.597Z","trace_propagation_headers":{"sentry-trace":"5d70842081f045d3b8986a63683c0740-87376907679c46a2","baggage":"sentry-trace_id=5d70842081f045d3b8986a63683c0740,sentry-release=Unknown"},"correlation_id":"01KWPJ87VD2ZGYBHN576S02EVH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:30.598Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-43fdf99ec1c4d24b78f2c3eb: start","job_status":"start","queue_duration_s":0.415131,"scheduling_latency_s":0.415152}2288vm-test-run-nixbot-gitlab> gitlab # [  230.817327] sidekiq-cluster[2150]: {"severity":"INFO","time":"2026-07-04T12:39:31.030Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"43fdf99ec1c4d24b78f2c3eb","created_at":"2026-07-04T12:39:30.597Z","trace_propagation_headers":{"sentry-trace":"5d70842081f045d3b8986a63683c0740-87376907679c46a2","baggage":"sentry-trace_id=5d70842081f045d3b8986a63683c0740,sentry-release=Unknown"},"correlation_id":"01KWPJ87VD2ZGYBHN576S02EVH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-07-04T12:39:30.598Z","job_size_bytes":3,"pid":2150,"sidekiq_tid":"mada","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-43fdf99ec1c4d24b78f2c3eb: done: 0.017011 sec","job_status":"done","queue_duration_s":0.415131,"scheduling_latency_s":0.415152,"redis_calls":7,"redis_duration_s":0.0018059999999999999,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001232,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000574,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010871,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017011,"completed_at":"2026-07-04T12:39:31.030Z","load_balancing_strategy":"primary","db_duration_s":0.001839,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2289vm-test-run-nixbot-gitlab> gitlab # [  230.842580] nix-daemon[3304]: accepted connection from pid 3735, user nixbot2290vm-test-run-nixbot-gitlab> gitlab # [  230.949395] gitaly[1392]: time="2026-07-04T12:39:31.899Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPJ88KSQGJM9G5HXBH7GZQW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T12:40:01.898" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T12:39:31.899" grpc.time_ms=0.644 pid=1392 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2291vm-test-run-nixbot-gitlab> gitlab # [  230.968785] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/04:12:39:31 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/98211a6541adb8d86116a7680ad181c0d8549f29/statuses HTTP/1.1" 200 1328 "" "curl/8.20.0" 12372292vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/98211a6541adb8d86116a7680ad181c0d8549f29/statuses, in 1.39 seconds)2293vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': '98211a6541adb8d86116a7680ad181c0d8549f29', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-04T12:39:17.921Z', 'started_at': None, 'finished_at': '2026-07-04T12:39:25.364Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '98211a6541adb8d86116a7680ad181c0d8549f29', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-07-04T12:39:25.557Z', 'started_at': None, 'finished_at': '2026-07-04T12:39:30.059Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2294vm-test-run-nixbot-gitlab> (finished: subtest: eval and build post commit statuses, in 14.13 seconds)2295vm-test-run-nixbot-gitlab> (finished: run the VM test script, in 266.42 seconds)2296vm-test-run-nixbot-gitlab> test script finished in 266.45s2297vm-test-run-nixbot-gitlab> cleanup2298vm-test-run-nixbot-gitlab> kill QemuMachine (pid 13)2299vm-test-run-nixbot-gitlab> gitlab # qemu-system-x86_64: terminating on signal 15 from pid 6 (/nix/store/60m4rxhg2fldqaak400c0lry96ijrzqn-python3-3.13.13/bin/python3.13)2300vm-test-run-nixbot-gitlab> (finished: cleanup, in 1.25 seconds)23012302post-build step Upload coverage to codecov: ok2303Skipping codecov: project=Mic92/nixbot attr=x86_64-linux.nixbot-gitlab